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ABSTRACT 


This  thesis  addresses  software  evolution  from  the  perspective  of  standards 
interoperability.  We  address  the  issue  of  how  to  apply  contemporary  software  safety 
assurance  standards  to  legacy  safety-critical  systems,  with  the  aim  of  recertifying  the 
legacy  systems  to  the  contemporary  standards.  The  application  of  RTCA  DO-178B 
‘Software  Considerations  in  Airborne  Systems  and  Equipment  Certification’  to  modified 
legacy  software  is  the  primary  focus  of  this  thesis.  We  present  a  model  to  capture  the 
relationships  between  pre-  and  post-modification  software  and  standards.  The  proposed 
formal  model  is  then  applied  to  the  requirements  for  RTCA  DO-178B  and  MIL-STD-498 
as  representative  examples  of  contemporary  and  legacy  software  standards.  The  results 
provide  guidance  on  how  to  achieve  airworthiness  certification  for  modified  legacy 
software,  whilst  maximizing  the  use  of  software  products  from  the  previous  development. 


V 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


VI 


TABLE  OF  CONTENTS 


1.  INTRODUCTION . 1 

A.  EVOLUTIONARY  DEVELOPMENT  OF  AIRCRAFT  AND 

AEROSPACE  SOFTWARE . 1 

B.  SAFETY-CRITICAL  SOFTWARE  CERTIFICATION . 4 

C.  UNDERLYING  CONCEPTS  AND  DEFINITIONS . 6 

1.  Legacy  Software . 6 

2,  Software  Safety  within  System  Safety . 6 

3.  Software  Assurance . 8 

4,  Mission  Hazards . 9 

IL  SAFETY-CRITICAL  SOFTWARE  STANDARDS . 11 

A,  EXAMPLES  FROM  THE  AEROSPACE  DOMAIN . 11 

B,  EXAMPLES  FROM  OTHER  DOMAINS . 11 

C.  RTCA  DO-178B  -  SOFTWARE  CONSIDERATIONS  IN 

AIRBORNE  SYSTEMS  AND  EQUIPMENT  CERTIFICATION . 12 

1.  DO-178B  Fault  Condition  Categories  and  Safety  Levels  for 

Software . 12 

2.  DO-178B  Objectives,  Activities,  Considerations  and  Evidence . 13 

3.  Circumstances  for  the  Application  of  DO-178B . 14 

D.  MIL-STD-498  -  SOFTWARE  DEVELOPMENT  AND 

DOCUMENTATION . 15 

1,  Background  and  Scope  of  MIL-STD-498 . 15 

2.  MIL-STD-498  Process  and  Product  Requirements . 16 

3.  MIL-STD-498  and  Safety  Requirements . 18 

4,  MIL-STD-498  Software  Product  Evaluation  and  Quality 

Assurance . 19 

III.  SOFTWARE  EVOLUTION,  CERTIFICATION  AND  THE 

RELATIONSHIP  WITH  SOFTWARE  STANDARDS . 21 

A.  SOFTWARE  EVOLUTION  AS  REENGINEERING . 21 

B.  SOFTWARE  PRODUCT  VERIFICATION . 22 

C.  SOFTWARE  EVOLUTION  AND  CERTIFICATION . 23 

1.  Using  Deductive  Logical  for  Software  Certification . 23 

2.  Establishing  the  Safety  Level  Assessment . 24 

3,  Carrying  Out  Software  Evolution . 25 

4,  Achieving  Airworthiness  Certification . 25 

D.  SOFTWARE  EVOLUTION  AND  STANDARDS . 25 

E.  ABSTRACT  ALGEBRA  AND  MORPHISMS . 28 

F.  ABSTRACT  ALGEBRA  AND  SOFTWARE  DEVELOPMENT . 29 

G.  ABSTRACT  ALGEBRA  AND  SOFTWARE  EVOLUTION . 32 

H.  APPLICATION  OF  MORPHISM  TO  SOFTWARE  EVOLUTION . 33 

1.  Previous  Software  Development . 35 

vii 


2,  Software  Modification . 36 

3,  Software  Product  Morphism . 38 

a.  Code  Traceability . 39 

b.  Software  Testing. . 42 

c.  General  Considerations  for  Software  Product  Morphism . 44 

IV.  RELATED  WORK . 45 

A,  ARCHITECTURAL  TRANSFORMATION . 45 

1,  Hierarchical  Typed  Hypergraphs . 45 

2,  Unified  Modeling  Language  for  Real-Time . 46 

3,  Hierarchical  Typed  Hypergraphs  Transformations . 46 

4,  Evaluation  of  Quality  Characteristics . 48 

B,  COMPUTER-AIDED  SOFTWARE  EVOLUTION . 48 

C,  F/RF-lllC  AGM-142E/1760  INTEGRATION . 50 

V.  CONCLUSION . 53 

A,  KEY  FINDINGS  AND  ACCOMPLISHMENTS . 53 

B,  CONCLUDING  REMARKS . 55 

C,  FUTURE  WORK . 56 

1.  Determine  the  Morphisms  Required  for  Activities  and 

Products . 56 

2.  Case  Studies . 56 

3.  Automation . 56 

4.  Different  Comhinations  of  Other  Standards . 56 

5.  Application  to  Other  Domains  and  Dimensions . 57 

6.  Cost-Benefit  Analysis  With  Respect  to  Software/System  Age . 57 

APPENDIX:  EXTRACTS  FROM  SELECTED  STANDARDS . 59 

A,  EXTRACTS  FROM  RTCA  DO-178B . 59 

B,  EXTRACTS  FROM  MIL-STD-498 . 68 

C,  EXTRACT  FROM  DATA  ITEM  DESCRIPTION  DI-IPSC-81433 

(SOFTWARE  REQUIREMENTS  SPECIFICATION) . 69 

LIST  OF  REFERENCES . 71 

INITIAL  DISTRIBUTION  LIST . 73 


viii 


LIST  OF  FIGURES 


Figure  1.  Model  for  Software  Reengineering  (From;  [1]) . 21 

Figure  2  Verifieation  Teehniques  (From;  [25]) . 23 

Figure  3.  Software  Evolution  and  Standards  Relationship . 26 

Figure  4.  Metaelass  Mapping  of  HTH  and  UML-RT  Elements  (Erom;  [28]) . 46 

Eigure  5.  Hypergraph  Graphieal  T-Notation  (Erom;  [28]) . 47 

Eigure  6.  Relational  Hypergraph  (Erom;  [29]) . 49 

Eigure  7.  AGM-142E  Integration  -  Simplified  Bloek  Diagram . 5 1 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


X 


LIST  OF  TABLES 


Table  1 .  RTCA  DO-178B  Safety  Categories  and  Software  Levels . 13 

Table  2.  Breakdown  of  Objectives  by  Lifecycle  Process  and  Safety  Level . 14 

Table  3.  Modified  Code  Traceability . 40 

Table  4.  Objectives,  Activities,  Outputs  and  Data  Control  Categories  (After;  15, 

Tables  A-1  through  A- 10) . 68 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


LIST  OF  EQUATIONS 


Equation  1 .  Software  Evolution . 27 

Equation  2.  Homomorphism  Example  -  Eogarithms . 29 

Equation  3.  Single  Produet  Development . 30 

Equation  4.  Single  Product  Development  -  Code  Example . 30 

Equation  5 .  Single  Activity . 31 

Equation  6.  Single  Activity  -  Coding  Example . 32 

Equation  7.  Software  Development . 32 

Equation  8.  Software  Evolution . 33 

Equation  9.  Identity  Morphism . 34 

Equation  10.  Partial  Morphism . 34 

Equation  1 1 .  Null  Morphism . 34 

Equation  12.  Code  Traceability . 35 

Equation  13.  Software  Testing . 36 

Equation  14.  Software  Coding  -  Modification . 37 

Equation  15.  Software  Code  Evolution . 37 

Equation  16.  Code  Traceability  -  Modification . 37 

Equation  17.  Software  Traceability  Evolution . 37 

Equation  18.  Software  Testing  -  Modification . 38 

Equation  19.  Software  Test  Results  Evolution . 38 

Equation  20.  Software  Configuration  Management  (CM)  Data  Evolution . 38 

Equation  21 .  Software  Trouble  Reports  Evolution . 38 

Equation  22.  Identity  Morphism  -  Code  Traceability . 41 

Equation  23.  Partial  Morphism  -  Design  Element  ‘B’  Code  Traceability . 41 

Equation  24.  Null  Morphism  -  Code  Traceability . 41 

Equation  25  Null  Morphism  -Test  Eogs  and  Results . 43 

Equation  26.  Partial  Morphism  -  Design  Element  ‘D’  Test  Eogs  and  Results . 43 


xiii 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


XIV 


LIST  OF  ABBREVIATIONS,  ACRONYMS,  AND  SYMBOLS 


Term 

Definition 

ADF 

Australian  Defenee  Foree 

ANSI 

Ameriean  National  Standards  Institute 

ARP 

Aerospaee  Reeommended  Praetiee 

CM 

Configuration  Management 

COM 

Computer  Operation  Manual 

COTS 

Commereial  off  the  shelf 

CPM 

Computer  Programming  Manual 

CSCI 

Computer  Software  Configuration  Item 

DBDD 

Database  Design  Deseription 

DBF  STAN 

Defenee  Standard  (UK) 

DID 

Data  Item  Deseription 

ECSS 

European  Cooperation  for  Spaee  Standardization 

EUROCAE 

European  Organisation  for  Civil  Aviation  Equipment 

F 

Fighter 

F/A 

Fighter/ Attaek 

FAA 

Federal  Aviation  Administration 

FSM 

Firmware  Support  Manual 

HTH 

Hierarehieal  Typed  Hypergraph 

lAW 

In  aeeordanee  with 

IDD 

Interfaee  Design  Deseription 

lEC 

International  Eleetroteohnieal  Commission 

IEEE 

Institute  of  Eleetrieal  and  Eleetronie  Engineers 

IRS 

Interfaee  Requirements  Speeifieation 

ISO 

International  Organization  for  Standardization 

JHMCS 

Joint  Helmet  Mounted  Cueing  System 

JSSSC 

Joint  Software  System  Safety  Committee 

MC 

Mission  Computer 

MIE-HDBK 

Military  Handbook  (U.S.) 

XV 


Term 

Definition 

MIL-STD 

Military  Standard  (U.S.) 

MISRA 

Motor  Industry  Software  Reliability  Assoeiation 

MoD 

Ministry  of  Defenee 

NASA 

National  Aeronauties  and  Spaee  Administration 

OCD 

Operational  Coneept  Deseription 

OFF 

Operational  Flight  Programs 

PSAC 

Plan  for  Software  Aspeets  of  Certifieation 

RAAF 

Royal  Australian  Air  Force 

ROI 

Return  On  Investment 

RTCA 

Radio  Technical  Commission  for  Aeronautics 

S/W,  Sw 

Software 

SAE 

Society  of  Automotive  Engineers 

SCMP 

Software  Configuration  Management  Plan 

SCOM 

Software  Center  Operator  Manual 

SCS 

Software  Configuration  Set 

SDD 

Software  Design  Description 

SDP 

Software  Development  Plan 

SIOM 

Software  Input/Output  Manual 

SIP 

Software  Installation  Plan 

SLOC 

Source  lines  of  code 

SMP 

Stores  Management  Processor 

SPS 

Software  Product  Specification 

SQA 

Software  Quality  Assurance 

SQAP 

Software  Quality  Assurance  Plan 

SRS 

Software  Requirements  Specification 

SSDD 

System/Subsystem  Design  Description 

SSS 

System/Subsystem  Specification 

Std 

Standard 

STD 

Software  Test  Description 

STP 

Software  Test  Plan 

STR 

Software  Test  Report 

XVI 


Term 

Definition 

STrP 

Software  Transition  Plan 

SUM 

Software  User  Manual 

SVD 

Software  Version  Deseription 

SVP 

Software  Verifieation  Plan  (DO-178B) 

TAR 

Teehnieal  Airworthiness  Regulator  (ADF) 

TR 

Teohnieal  Report 

UK 

United  Kingdom  of  Great  Britain  and  Northern  Ireland 

UML-RT 

Unified  Modeling  Language  for  Real-Time 

U.S. 

United  States  of  Ameriea 

USMC 

United  States  Marine  Corps 

USN 

United  States  Navy 

A  (Alpha) 

Set  of  all  possible  symbols  defined  for  a  given  algebra 

Ax 

Superset  of  software  produets  for  version  X 

Bs,  Cs 

Subset  of  software  produets  of  Ax 

Rx 

Relationship  ‘x’  (between  software  and/or  standard) 

Sx 

Software  standard  ‘x’ 

X" 

nth  version  of  software  X 

A  single  software  produet 

<P 

Morphism  funetion 

A  single  software  development  aetivity 

Q 

Set  of  all  possible  operators  defined  for  a  given  algebra 

Produces 

M 

Set  of  all  real  numbers 

Set  of  all  positive  real  numbers 

0 

Null  set 

u 

Set  union 

c; 

Subset 

n 

U 

s=\ 

Union  of  n  sets 

xvii 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


LIST  OF  SAFETY  AND  SOFTWARE  STANDARDS 
AND  GUIDELINES 


Designation 

Status 

Title 

ANSI/IEEE  Std  730 

2002 

Standard  for  Software  Quality  Assurance  Plans 

MoD  Def  Stan  00-54 

Superseded 

Requirements  for  Safety  Related  Electronic 

Hardware  in  Defence  Equipment  (UK) 

MoD  Def  Stan  00-55 

Superseded 

Requirements  for  Safety  Related  Software  in 

Defence  Equipment  (UK) 

MoD  Def  Stan  00-56 

Interim 

Issue  3 

Safety  Management  Requirements  for  Defence 
Systems  (UK) 

MoD  Def  Stan  00-58 

Superseded 

HAZOP  Studies  on  Systems  Containing 
Programmable  Electronics  (UK) 

DEF(AUST)  5679 

1998 

The  Procurement  of  Computer-Based  Safety  Critical 
Systems 

DI(AF)  AAP  700  E  054 

2004 

Airworthiness  Design  Requirements  Manual 

EN  50126 

1999 

Railway  Applications  -  The  Specification  and 
Demonstration  of  Reliability,  Availability, 
Maintainability  and  Safety 

EN  50128 

2001 

Software  for  Railway  Control  and  Protection 

Systems 

FAA  Order  8110.49 

2003 

Software  Approval  Guidelines  (U.S.) 

H  ProgSakE 

2001 

Handbook  for  Software  in  Safety  Critical 

Applications  (Sweden) 

H  SystSakE 

System  Safety  Activities  for  Defense  Systems 
(Sweden) 

lEC  60880 

1986 

Software  for  Computers  in  the  Safety  Systems  of 
Nuclear  Power  Stations 

lEC  60880-2 

2000 

Software  for  Computers  Important  to  Safety  for 
Nuclear  Power  Plants  -  Part  2:  Software  Aspects  of 
Defence  Against  Common  Cause  Failures,  use  of 
Software  Tools  and  of  Pre-Developed  Software 

lEC  61508 

1998 

Functional  Safety  of 

Electrical/Electronic/Programmable  Electronic 
Safety-Related  Systems 

IEEE  Std  829 

1998 

Standard  for  Software  Test  Documentation 

IEEE  Std  1012 

2004 

Standard  for  Software  Verification  and  Validation 

IEEE  Std  1028 

1997 

Standard  for  Software  Reviews 

XIX 


Designation 

Status 

Title 

IEEE  Std  1228 

1994 

Standard  for  Software  Safety  Plans 

IEEE  Std  1298 

1992 

Software  Quality  Management  System 

lEEE/EIA  12207 

1996/97 

Industry  Implementation  of  ISO/IEC  12207-1995 
Information  Technology  -  Software  Eife  Cycle 
Processes 

ISO  9000-3 

1997 

Quality  Management  and  Quality  Assurance 

Standards  -  Guidelines  for  the  Application  of  ISO 

900 1 : 1 994  to  the  Development,  Supply,  and 
Maintenance  of  Computer  Software 

ISO/IEC  15026 

1998 

Information  technology  -  System  and  software 
integrity  levels 

ISO/IEC  TR  15942 

2000 

Information  Technology  -  Programming  languages 
-  Guide  for  the  Use  of  the  Ada  Programming 
Eanguage  in  High  Integrity  Systems 

JSSSC  SSSH 

1999 

Software  System  Safety  Handbook  (U.S.) 

MlE-HDBK-286 

A  Guide  for  DOD-STD-2168  (U.S.) 

MIE- STD-2 167A 

Cancelled 

Defense  System  Software  Development  (U.S.) 

MlE-STD-498 

1994 

Software  Development  and  Documentation  (U.S.) 

M1E-STD-882C 

Superseded 

System  Safety  Program  Requirements  (U.S.) 

MIE-STD-882D 

2000 

Standard  Practice  for  System  Safety  (U.S.) 

RTCA  DO-178B 

EUROCAE  ED-12B 

1992 

Software  Considerations  in  Airborne  Systems  and 
Equipment  Certification 

RTCA  DO-248B 

EUROCAE  ED-94B 

2001 

Final  Report  for  Clarification  of  DO-178B 
“Software  Considerations  in  Airborne  Systems  and 
Equipment  Certification” 

SAE  ARP  4754 

1996 

Certification  Considerations  for  Highly-Integrated 
or  Complex  Aircraft  Systems 

SAE  ARP  4761 

1996 

Guidelines  and  Methods  for  Conducting  the  Safety 
Assessment  Process  on  Civil  Airborne  Systems  and 
Equipment 

UE  1998 

1998 

Standard  for  Safety-Related  Software  (U.S.) 

XX 


GLOSSARY 


Term 

Definition 

Source 

Arity 

The  number  of  arguments  a  function  or 
operator  takes. 

computing-dictionary 

Quality 

Assurance 

(1)  A  planned  and  systematic  pattern  of  all 
actions  necessary  to  provide  adequate 
confidence  that  an  item  or  product  conforms 
to  established  technical  requirements. 

(2)  A  set  of  activities  designed  to  evaluate 
the  process  by  which  products  are 
developed  or  manufactured. 

IEEE610.12 

Safety 

Assurance 

(1)  The  degree  of  confidence  required  in  the 
correctness  of  a  particular  process  or  tool. 

(2)  The  planned  and  systematic  actions 
necessary  to  provide  adequate  confidence 
and  evidence  that  a  product  or  process 
satisfies  given  requirements. 

(1) DEE  STAN  00-55 

(2)  DO-178B 

Software 

Assurance 

Software  assurance  is  the  planned  and 
systematic  set  of  activities  that  ensures  that 
software  processes  and  products  conform  to 
requirements,  standards,  and  procedures. 
‘Processes’  include  all  of  the  activities 
involved  in  designing,  developing, 
enhancing,  and  maintaining  software; 
‘products’  include  the  software,  associated 
data,  its  documentation,  and  all  supporting 
and  reporting  paperwork. 

AAP  7001.054 

Software 

Quality 

The  ability  of  software  to  satisfy  its 
specified  requirements. 

MIE-STD-498 

System 

Safety 

The  application  of  engineering  and 
management  principles,  criteria,  and 
techniques  to  achieve  acceptable  mishap 
risk,  within  the  constraints  of  operational 
effectiveness  and  suitability,  time,  and  cost, 
throughout  all  phases  of  the  system  life 
cycle. 

MIE-STD-882D 

XXI 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


ACKNOWLEDGMENTS 


I  wish  to  express  my  sineere  gratitude  to  the  Royal  Australian  Air  Foree  for 
giving  me  the  opportunity  to  pursue  the  Master  of  Seience  in  Software  Engineering  at  the 
Naval  Postgraduate  Sehool.  In  a  elimate  of  searee  resourees,  the  opportunity  to  pursue 
edueational  opportunities  in  a  foreign  eountry  is  rare.  I  am  also  grateful  that  the  United 
States  Navy  and  the  Naval  Postgraduate  Sehool  have  the  wisdom  to  offer  this  degree 
program,  seeing  value  in  this  program  of  study  when  other  institutions  do  not. 

I  must  express  my  gratitude  to  Professor  Miehael,  my  aeademie  assoeiate  and 
thesis  advisor,  for  sharing  his  knowledge  and  experienee  with  me.  He  was  always  willing 
to  offer  his  insight  and  adviee  on  this  thesis  and  other  eoursework,  despite  the 
eonsiderable  eonstraints  on  his  time.  He  also  deserves  speeial  recognition  for  regularly 
stepping-in  to  fill-the-gap  when  a  course  required  an  instructor.  To  him,  and  the  other 
members  of  the  Software  Engineering  Department,  I  owe  a  great  debt. 

This  thesis  is  dedicated  to  my  wife  who  resolutely  endured  the  trials  of  being  a 
study-widow,  again,  and  provided  the  unfailing  support  that  permitted  me  to  focus  during 
the  duration  of  the  course. 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


XXIV 


I.  INTRODUCTION 

A,  EVOLUTIONARY  DEVELOPMENT  OF  AIRCRAFT  AND  AEROSPACE 

SOFTWARE 

There  are  many  reasons  for  software  evolution.  Seacord,  Plakosh  and  Lewis  [1] 
identify  three  categories  of  evolution:  (i)  maintenance,  (ii)  modernization  and  (iii) 
replacement.  They  describe  maintenance  as  small  changes  that  are  typically  corrections 
to  software  faults  or  minor  enhancements.  Modernization  involves  major  changes  to  a 
system,  but  which  preserve  a  significant  amount  of  the  old  system.  Modernization  may 
take  the  form  of  retargeting  old  software  to  a  new  hardware  platform;  revamping  the 
human  machine  interface  to  improve  usability;  component  substitution,  such  as  with 
alternate  commercial  products;  source  code  translation  to  new  versions  of  the  same 
language  or  different  languages  to  that  previously  used;  code  reduction  to  remove  unused 
functionality  or  re-factor  remaining  functionality;  and  functional  transformation  to 
achieve  structural  improvement.  Replacement  involves  adopting  a  completely  new 
design  for  a  system  when  the  old  system  cannot  be  modernized  in  an  effective  manner. 

Seacord,  Plakosh  and  Lewis  go  on  to  identify  complexity,  software  technology 
and  engineering  processes,  risk,  commercial  components,  and  changed  business 
objectives  as  challenges  to  modernization.  Leveson  [2]  identifies  the  appearance  of  new 
hazards,  an  increased  exposure  to  software-intensive  systems,  greater  amounts  of  energy 
being  monitored  or  controlled  by  software,  and  an  increased  reliance  on  software  for 
monitoring  and  control  of  systems  as  further  challenges  to  software  maintenance. 
Additional  challenges  experienced  in  the  military  domain  include  the  need  for  high 
degrees  of  inter-operability  with  external  systems;  national  and  international  rather  than 
personal  or  commercial  security  concerns;  and  a  harsh  operating  environment  (i.e., 
combat)  in  which  the  system  must  continue  to  operate. 

Military  aerospace  systems  are  examples  of  software-intensive  systems  that 
exhibit  many  of  the  aforementioned  challenges.  Such  systems  are  costly  to  produce  and 
take  many  years  to  develop.  For  example,  consider  the  following  timeline  for  the  Boeing 
(McDonnell  Douglas)  F/A-18A/B/C/D:  [3,4,5, 6] 


1 


1970s 

1975 

1970/1980S 

1978  &  1979 

1980-1988 

1982-1988 

1984-1990 

1986- 1990 

1987- 2000 

1991- 1993 

1992- 2008 

1995- 2000 

1996- 1999 
1997 
1997 

1999  onwards 
2002-2010 
2002-2009 

2004-2008 

2007-2014 

2015 

2017-2020 

2020 

2025 


Predecessor  design  (Northrop  YF-17)  proposed  as  an  Air 
Combat  Fighter  for  the  United  States  Air  Force.  (F-16  chosen 
instead). 

Modified  design  (F-18)  accepted  as  a  Naval  Air  Combat  Fighter 
for  the  United  States  Navy  (USN)  and  Marine  Corps  (USMC). 

Variant  designs  for  Attack  (A- 18)  and  Trainer  (TF-18)  versions 
developed  but  eventually  merged  to  become  the  F/A-18A  single¬ 
seat  and  F/A-18B  dual  seat  versions. 

First  flights  of  an  F/A-18A  &  B  respectively. 

F/A-18A  &  B  aircraft  delivered  to  USN  and  USMC. 

CF-18A  &  B  aircraft  delivered  to  Canadian  Forces  -  Air 
Command. 

AF/A-18A  &  B  aircraft  delivered  to  Royal  Australian  Air  Force 
(RAAF).  [7] 

EF-18A  &  B  aircraft  delivered  to  Spanish  Air  Force. 

F/A-18C  &  D  delivered  to  USN  and  USMC. 

F/A-18C  &  D  aircraft  delivered  to  Kuwait  Air  Force. 

Upgrade  of  Spanish  Air  Force  EF-18A  &  B  aircraft. 

E-18C  &  D  aircraft  delivered  to  Einnish  Air  Eorce.  [8] 

E/A-18C  &  D  aircraft  delivered  to  Swiss  Air  Eorce. 

Delivery  of  E/A-18D  aircraft  to  Royal  Malaysian  Air  Eorce. 

Merger  of  the  Boeing  and  McDonnell  Douglas  companies. 

Upgrade  of  USN  and  USMC  E/A-18A,  B,  C  &  D  aircraft. 

Upgrade  of  RAAF  F/A-18A  &  B  aircraft.  [7] 

Upgrade  of  Canadian  Forces  -  Air  Command  CF-18A  &  B 
aircraft.  [9] 

Upgrade  of  Swiss  Air  Force  F/A-18C  &  D  aircraft. 

Upgrade  of  Finnish  Air  Force  F-18C  &  D  aircraft.  [8] 

Planned  withdrawal  of  RAAF  AF/A-18  aircraft.  [10] 

Planned  withdrawal  of  the  Canadian  Forces  -  Air  Command  CF- 
18. [9] 

Planned  withdrawal  of  Spanish  Air  Force  EF/A-18  aircraft 
Planned  withdrawal  of  Finnish  Air  Force  F-18  aircraft.  [8] 
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Military  aircraft  are  not  alone  in  their  long  life  and  eontinual  upgrade.  After  three 
years  of  initial  design  work  on  the  original  design,  the  Boeing  747-100  design  was 
aeeepted  in  1966  and  entered  service  in  1970.  In  the  forty  years  sinee  then,  four  other 
signifieant  variants  and  thirteen  minor  variants  have  been  or  are  being  built  [11].  The 
delivery  of  the  latest  747-8  aireraft  is  eonjeetured  to  last  twenty  years  from  a  planned 
serviee  date  of  2009.  It  is  eoneeivable  that  these  aireraft  will  be  flown  for  twenty  years 
beyond  final  delivery.  While  the  last  design  will  be  substantially  different  from  the  first, 
this  represents  around  eighty  years  of  evolution. 

The  F/A-18  timeline  above  only  ineludes  four  major  variants  of  the  F/A-18 
without  mention  of  the  different  eonfigurations  of  equipment  and  software  for  the  eight 
nations  that  utilize  this  aircraft.  Nor  does  it  inelude  the  three  F/A-18E/F/G  variants 
whieh  some  eonsider  to  be  substantially  different  from  the  earlier  variants  of  the  aircraft. 
The  timeline  above  reveals  an  approximately  thirty  year  life  to  date  and  around  fifty  years 
total  for  development,  maintenanee,  and  modernization  from  the  coneeption  of  the  F/A- 
18  to  its  planned  final  disposal.  Two  dominant  reasons  for  ehanges  to  aireraft  are  new 
mission  requirements  and  teehnology  improvements,  sueh  as  the  addition  of  an  attaek 
role  as  well  as  the  air  eombat  role  for  the  F/A-18,  and  the  availability  of  new  equipment 
sueh  as  the  Joint  Helmet  Mounted  Cueing  System  (JHMCS)  or  new  weapons  sueh  as  the 
AIM-9X  air-to-air  missile.  Given  the  long  development  time  and  serviee  history  of 
aireraft,  many  ehanges  to  an  aireraft  design  ean  be  expeeted  over  its  lifespan. 
Furthermore,  the  eonsiderable  amount  of  previous  expenditure  on  an  existing  aireraft 
invites  modernization  of  the  existing  platform  before  purehasing  a  new  aireraft.  Unit 
eosts  of  F/A-18A  &  B  aireraft  have  been  reported  between  USD28-35  million. 

A  eritieal  enabler  for  variants  and  upgrades  of  all  aireraft  is  software.  The 
eolleetion  of  Operational  Flight  Programs  (OFP)  in  the  many  different  proeessors  of  the 
F/A-18  is  eollectively  ealled  a  Software  Configuration  Set  (SCS).  Some  of  the  SCS  that 
have  been  developed  or  are  currently  in  development  or  planning  for  the  F/A-18A/B/C/D 
include  89C,  91C,  92A,  09C,  lOA,  IIC,  12A,  13C,  15C,  17C,  18E,  19C,  21C,  23C,  25C. 
Some  of  these  SCS  were  integral  to  the  hardware  upgrade  programs  that  were  listed  in 
the  timeline.  However,  most  of  them  are  new  versions  of  an  SCS  for  the  same  target 
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platform.  In  addition  to  this  list  of  SCS,  there  are  also  different  versions  of  some  of  the 
above  SCS  for  different  international  customers,  for  example,  15C  for  the  USN  and 
USMC  but  15CA  for  the  RAAF;  and  at  least  two  countries  outside  the  United  States 
(U.S.)  are  both  maintaining  and  modernizing  the  SCS  that  they  receive  to  meet  their  own 
unique  requirements  and  priorities.  The  15C  SCS  is  a  recent  software  development  that 
demonstrates  many  of  the  challenges  to  software  modernization.  The  15C  SCS  was 
delivered  in  2001  after  four  years  of  development.  This  SCS  started  out  with  seventy- 
five  high-level  Statements  of  Requirements  to  be  completed  in  three  builds  and  ended 
with  134  Statements  of  Requirements  delivered  over  four  builds.  The  final  product 
integrated  three  new  weapons  and  five  new  major  avionics  systems.  The  15C  SCS  has 
over  ten  million  source  lines  of  code  (SLOC),  across  more  than  forty  processors  and  uses 
twelve  languages  in  the  aircraft  with  a  further  two  in  the  development  environment  which 
itself  has  four  million  SLOC  [12].  Several  computer  processors  in  the  F/A-18  required 
upgrading,  forcing  retargeting  of  the  OFF  to  run  on  them.  Replacement  color  displays 
and  the  integration  of  the  JHMCS  have  enabled  a  revamping  of  parts  of  the  human 
machine  interface. 

B,  SAFETY-CRITICAL  SOFTWARE  CERTIFICATION 

An  essential  part  of  developing  safety-related  aerospace  software  that  is  expected 
to  be  operationally  fielded  is  to  comply  with  the  airworthiness  design  requirements 
specified  by  the  certification  authority.  Examples  of  airworthiness  certification 
authorities  include  the  Federal  Aviation  Administration  (FAA)  for  civilian  aviation  in  the 
U.S.,  the  Australian  Defence  Force  (ADF)  for  military  aviation  in  Australia,  and  the 
Naval  Air  Systems  Command  for  USN  and  USMC  aviation.  Airworthiness  design 
requirements  address  the  acceptable  level  of  confidence  required  in  the  safety  of  all  parts 
of  the  aircraft  system:  hardware,  software  and  the  human  operators  (sometimes  referred 
to  as  “skinware”).  Software  system  safety  requirements  address  those  parts  of  a  system 
for  which  software  is  identified  as  the  source  of,  detector  of,  or  means  of  containing  a 
system  fault,  regardless  of  the  locus  of  the  fault.  “Certification  is  normally  based  on  the 
use  of  some  form  of  standard...”  [13].  The  current  ADF  preference  for  a  software 
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assurance  standard  for  the  development  of  safety-related  aerospaee  software  is  the  Radio 
Teehnioal  Commission  for  Aeronauties  (RTCA)  DO-178B  ‘Software  Considerations  in 
Airborne  Systems  and  Equipment  Certifieation’  [14], 

Obtaining  an  appropriate  return  on  investment  (ROI)  is  an  understandable 
expeetation  for  the  aequirer  of  any  system.  The  eonsiderable  amount  of  resourees  it  takes 
to  develop  a  large  aerospaee  system  neeessitates  a  relatively  lengthy  time  of  system 
operation  in  order  to  reeoup  this  investment.  The  system  will  ehange  over  time  to 
aeeount  for  one  or  more  of  the  following:  eorreetions  to  design  faults  or  implementation 
flaws  in  the  previous  system,  adaptations  of  existing  system  funetions  to  aeeommodate 
ehanges  in  environment  or  operations,  or  eompletely  new  features  that  meet  previously 
infeasible  or  unimagined  requirements.  One  element  for  eonsideration  when  modifying 
software  is  to  maintain  at  least  an  equivalent  level  of  assuranee  as  that  for  the  initial 
development.  However,  legaey  software  previously  developed  to  standards  sueh  as 
DOD-STD-2167A  or  MIL-STD-498  may  not  have  ineluded  adequate  provision  for 
software  assuranee  that  would  meet  today’s  standards.  As  sueh,  a  desirable  goal  during 
software  modifieation  is  to  upgrade  the  previous  eertifieation  basis  by  addressing  the 
objeetives  of  a  eontemporary  software  assuranee  standard  sueh  as  DO-178B  [15]. 
Applying  DO-178B  guidanee  to  the  development  of  new  software  requires  eonsiderable 
effort  at  safety  levels  of  higher  integrity,  but  is  relatively  straightforward  when  eompared 
with  the  re-eertifieation  of  legaey  software  to  DO-178B  that  did  not  have  the  DO-178B 
guidelines  applied  during  the  previous  software  development. 

Developers  may  ehoose  to  improve  their  software  development  proeesses  when 
modifying  legaey  software  in  order  to  aehieve  eertifieation  to  a  new  software  assuranee 
standard.  When  an  applieant  seeks  re/eertifieation,  the  eertifieation  authority  takes  into 
aeeount  whether  the  evidenee  provided  by  the  software  development  team  suffieiently 
addresses  the  software  assuranee  objeetives,  aetivities  and  eonsiderations.  The  level  of 
eonfidenee  one  has  that  the  modified  software  deserves  eertifieation  against  a  new 
software  assuranee  standard  should  inerease  eaeh  time  the  legaey  software  is  further 
modified  and  re-eertifieation  is  subsequently  aehieved,  ceteris  paribus.  This  approaeh  to 
eertifieation  of  evolving  software  is  the  eurrent  praetiee  of  the  ADF. 
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However,  re-certification  of  software  to  a  new  software  assurance  standard  may 
be  built  upon  legacy  software  that  in  some  fundamental  way  does  not  warrant 
certification  to  a  new  standard  of  level.  Whilst  the  improved  processes  applied  during 
modification  address  the  modification  itself  and  any  identified  interfaces  to  non-modified 
software,  the  processes  may  not  address  the  fundamental  system  safety  properties  of  that 
part  of  the  underlying  legacy  software  that  is  not  addressed  in  the  modification  or  is  not 
identified  as  interfacing  the  modified  areas.  This  approach  to  re-certifying  legacy 
software  raises  the  following  question:  Could  legacy  software  be  fundamentally  flawed 
in  areas  that  are  left  unmodified  during  software  evolution  and  result  in  unwarranted 
certification  of  software  to  a  new  software  assurance  standard? 

C.  UNDERLYING  CONCEPTS  AND  DEFINITIONS 

1.  Legacy  Software 

For  the  purposes  of  this  thesis,  the  term  legacy  software  means  software  that  has 
been  previously  developed  and  is  subject  to  modification,  that  is,  both  maintenance  and 
modernization.  More  specifically,  it  is  software  that  has  been  developed  to  a  defined 
standard,  or  through  a  defined  process,  so  that  the  software  has  a  known  pedigree,  but  a 
pedigree  that  is  not  currently  desirable.  It  is  recognized  that  this  is  a  narrow  definition 
and  does  not,  amongst  other  scenarios,  include  the  reuse  of  software  in  a  new  application 
without  first  being  modified. 

2.  Software  Safety  within  System  Safety 
MIL-STD-882D  has  the  following  definition  for  safety: 

Freedom  from  those  conditions  that  can  cause  death,  injury,  occupational 
illness,  damage  to  or  loss  of  equipment  or  property,  or  damage  to  the 
environment.  [16] 

Leveson  defines  safety  in  a  manner  consistent  with  this  absolute  point  of  view  as 
“freedom  from  accidents  or  losses,”  but  recognizes  that  this  is  not  achievable  for  real- 
world  systems,  especially  those  systems  that  are  complex.  Absolute  safety  is,  however, 
the  goal  that  should  be  the  starting  point  from  which  judgments  about  acceptable  levels  of 
mishap  risk  are  made.  Leveson  goes  on  to  make  the  case  that  safety  is  a  system  property 
that  has  a  contribution  from  software  whenever  software  is  involved  in  a  system.  The 
definition  for  system  safety  from  MIL-STD-882D  is: 
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The  application  of  engineering  and  management  principles,  criteria  and 
techniques  to  achieve  acceptable  mishap  risk  within  the  constraints  of 
operational  effectiveness,  time  and  cost  throughout  the  system’s  life  cycle. 

This  definition  and  its  nearly  identical  predecessor  in  MIL-STD-882C  start  out 
with  a  provisional  view  of  system  safety  that  is  a  function  of  system  effectiveness  and 
development  schedule  and  cost. 

Roland  and  Moriarty  [17]  state  that  the  following  as  the  concept  for  system 

safety: 

...  involves  a  planned,  discipline,  systematically  organized,  and  before- 
the-fact  process  characterized  as  the  identify-analyze-control  method  of 
safety. 

In  both  of  the  preceding  definitions,  system  safety  is  assumed  to  be  reliant  on  the 
process  by  which  a  system  is  developed;  that  is,  system  safety  does  not  simply  happen  by 
chance,  but  is  instead  part  of  system  design.  However,  just  having  a  development 
process  does  not  guarantee  that  a  system  will  be  safe  (whatever  your  definition  of  safety). 
The  process  must  be  suitable,  rigorous,  complete  and  actually  used  to  develop  a  product 
that  can  be  regarded  as  relatively  safe  with  some  degree  of  confidence  in  the  assertion  of 
safety. 

The  application  of  system  safety  engineering  focuses  on  the  early  identification 
and  analysis  of  hazards  which  in  turn  permits  the  system  developer  to  mitigate  them 
through  system  design.  This  is  the  preferred  method  of  treating  systems  hazards.  The 
alternative  is  late  identification  of  hazards  which  forces  either  the  treatment  of  hazards 
through  the  less  desirable  procedural  and  training  mitigation  measures,  or  the  costly 
rework  (in  time  and  money)  to  incorporate  design  mitigation  measures. 

Software  is  an  abstraction  and  as  such,  it  has  no  substance  and  cannot  directly 
harm  people,  property  or  the  environment.  However,  software  can  be  responsible  for  the 
loss.  The  opportunity  for  software  to  contribute  to  loss  is  enabled  through  its  use  by 
system  developers  to  sense  and  control  physical  components  within  a  system  and  its 
environment;  the  physical  components  can  release  energy  that  can  harm  someone  or 
something.  Safety  of  software  is  just  one  of  a  system’s  properties,  and  is  dependent  on 
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the  software,  hardware,  operator  and  external  faetors.  Leveson  provides  the  following 
definition  of  software  system  safety  that  is  consistent  with  this  perspective:  “Software 
System  Safety  implies  that  the  software  will  execute  within  a  system  context  without 
contributing  to  hazards”  [2], 

3.  Software  Assurance 

The  ADF  Airworthiness  Design  Requirements  Manual  describes  software 
assurance  in  the  following  terms: 

Software  assurance  is  the  planned  and  systematic  set  of  activities  that 
ensures  that  software  processes  and  products  conform  to  requirements, 
standards,  and  procedures.  ‘Processes’  include  all  of  the  activities 
involved  in  designing,  developing,  enhancing,  and  maintaining  software; 
‘products’  include  the  software,  associated  data,  its  documentation,  and  all 
supporting  and  reporting  paperwork.  [14] 

The  Institute  of  Electrical  and  Electronic  Engineers  (IEEE)  Standard  Glossary  of 
Software  Engineering  Terminology  does  not  define  software  assurance  but  does  have  an 
entry  for  software  quality  assurance  which  refers  readers  to  quality  assurance  which 
states: 


(1)  A  planned  and  systematic  pattern  of  all  actions  necessary  to  provide 
adequate  confidence  that  an  item  or  product  conforms  to  established 
technical  requirements. 

(2)  A  set  of  activities  designed  to  evaluate  the  process  by  which  products 
are  developed  or  manufactured.  [18] 

The  National  Aeronautics  and  Space  Administration  (NASA)  does  define 
Software  Assurance,  and  does  so  in  terms  from  the  IEEE  Standard  Glossary  [18],  but 
adds  that  NASA’s  definition  includes  the  disciplines  of  software  quality,  safety, 
reliability,  and  verification  and  validation: 

The  planned  and  systematic  set  of  activities  that  ensure  that  software  life 
cycle  processes  and  products  conform  to  requirements,  standards,  and 
procedures.  [19] 

NASA  elaborates  on  this  definition  of  software  assurance  by  introducing 
functional  or  mission-requirement  elements  and  adds  the  aspect  of  oversight  to  the 
assurance  activity: 
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Software  assurance  is  an  umbrella  risk  mitigation  strategy  for  safety  and 
mission  assurance  of  all  of  NASA’s  software.  The  purpose  of  software 
assurance  is  to  assure  that  software  products  are  of  high  quality  and 
operate  safely. 

Assure  is  used  when  software  assurance  practitioners  make  certain  that  the 
specified  software  assurance,  management,  and  engineering  activities  have 
been  performed  by  others. 

Finally,  for  plain  English  definitions  of  assurance,  the  following  are  taken  from 
Wiktionary: 

(1)  The  act  of  assuring;  a  declaration  tending  to  inspire  full  confidence; 
that  which  is  designed  to  give  confidence. 

(2)  The  state  of  being  assured;  firm  persuasion;  full  confidence  or  trust; 
freedom  from  doubt;  certainty.  [20] 

Based  on  the  preceding  definitions,  one  can  conclude  that  the  aim  of  software 
assurance  is  to  provide  confidence  that  the  software  complies  with  all  of  its  requirements 
(e.g.,  functional,  safety,  reliability)  and  that  software  assurance  is  distinct  from  other 
software  development  activities.  In  some  cases  software  assurance  activities  require  an 
independent  oversight  to  justify  the  non-independent  claims  of  software  assurance. 

4,  Mission  Hazards 

An  additional  consideration  for  military  systems  is  mission-worthiness.  If  part  or 
all  of  a  combat  military  system’s  (e.g.,  tank,  ship,  aircraft)  self-defense  mechanisms  is 
inoperable,  the  system  will  be  subject  to  additional  mission  hazards  that  are  a  part  of  the 
combat  environment,  for  instance,  loss  of  lives  and  equipment  through  destruction  or 
capture.  Unacceptable  consequential  losses  may  also  be  incurred  by  either  of  the 
combatants  if  a  military  system’s  offensive  capability  is  faulty  or  inoperable,  such  as 
failure  to  destroy  an  enemy  may  result  in  subsequent  loss  of  friendly  forces,  or 
inaccuracies  in  targeting  may  increase  collateral  damage  and  loss.  These  examples  do 
not  fit  the  contemporary  view  of  safety  hazards,  but  rather  operational  or  performance 
failures  that  have  hazards  as  an  indirect  consequence.  However,  due  to  the  dire 
consequences  of  some  operational  or  performance  failures  in  a  combat  environment, 
application  of  the  techniques  that  assure  safety  of  software  can  also  be  applied  to  the 

mission  requirements  of  software-intensive  systems. 
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II. 


SAFETY-CRITICAL  SOFTWARE  STANDARDS 


A,  EXAMPLES  FROM  THE  AEROSPACE  DOMAIN 

The  focus  of  this  thesis  is  the  application  of  DO-178B  as  it  is  the  preferred 
standard  for  software  assurance  for  safety-related  airborne  software  in  the  ADF.  This  is 
not  to  say  that  alternative  standards  do  not  exist  or  are  unacceptable.  In  the  aerospace 
domain,  software-related  standards  in  use  include: 

RTCA  DO-178B  ‘Software  Considerations  in  Airborne  Systems  and  Equipment 
Certification’  in  conjunction  with  an  acceptable  system/software  safety 
standard  for  civilian  aviation  in  the  U.S. 

MIL-STD-498  ‘Software  Development  and  Documentation’  in  conjunction  with 
MIL-STD-882D  ‘Standard  Practice  for  System  Safety’  for  military 
aviation  in  the  U.S. 

DEE  STAN  00-55  ‘Requirements  for  Safety  Related  Software  in  Defence 
Equipment’!  produced  by  the  United  Kingdom  Ministry  of  Defence 

NASA-STD-8739.8  ‘Software  Assurance  Standard’  for  the  development  of 
aerospace  software  within  NASA 

ECSS  Q-80B  ‘Software  Product  Assurance’  by  the  European  Space  Agency 

B,  EXAMPLES  FROM  OTHER  DOMAINS 

Aerospace  software  is  just  one  safety-critical  domain  that  requires  standards  for 
software  development  and/or  assurance.  Other  domains  and  the  standards  proposed  for 
them  include: 

EN  50128  ‘Software  for  railway  control  and  protection  systems’  and  lEC  62279 
‘Software  for  railway  control  and  protection  systems’  for  the  development 
of  software  in  the  railway  domain 

lEC  60880  ‘Software  for  computers  in  safety  systems  of  nuclear  power  stations’ 
and  lEC  62138  ‘Software  aspects  for  computer-based  systems  performing 
category  B  or  C  functions’  for  application  to  nuclear  power  station 
software 

lEC  60601-1-4  ‘General  requirements  for  safety  -  Collateral  Standard: 
Programmable  electrical  medical  systems’  for  software  in  the  medical 
equipment  domain 


1  Now  obsolete  and  superceded  by  DBF  STAN  00-56  ‘Safety  Management  Requirements  for  Defence 
Systems’. 
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ISO/TR  15497  ‘Road  vehicles  -  Development  guidelines  for  vehicle  based 
software’  and  the  Motor  Industry  Software  Reliability  Association 
(MISRA)  Report  2  ‘Integrity’  for  software  in  the  road  vehicle  domain 

The  following  list  of  standards  has  been  proposed  for  general  use,  rather  than 
being  identified  for  application  in  a  single  domain; 

lEC  61508  ‘Functional  safety  of  electrical/electronic/programmable  electronic 
safety-related  systems’ 

ISO/IEC  12207  ‘Information  technology  -  Software  life  cycle  processes’ 

ISO/IEC  9126  ‘Software  engineering  -  Product  quality’ 

ISO/IEC  14598  ‘Software  engineering  -  Product  evaluation’ 

ISO/IEC  90003  ‘Software  engineering  -  Guidelines  for  the  application  of  ISO 
9001:2000  to  computer  software’ 

C.  RTCA  DO-178B  -  SOFTWARE  CONSIDERATIONS  IN  AIRBORNE 

SYSTEMS  AND  EQUIPMENT  CERTIFICATION 

1.  DO-178B  Fault  Condition  Categories  and  Safety  Levels  for  Software 

DO-178B  was  developed  in  collaboration  with  the  European  Organisation  for 
Civil  Aviation  Equipment  (EUROCAE)  which  published  the  document  as  ED-12B  with 
the  same  title.  The  RTCA  is  not  an  officially  sanctioned  authority,  and  as  such,  DO- 
178B  is  not  mandatory  for  use  within  the  U.S..  However,  DO-178B  is  highly  regarded 
within  the  aviation  community  and  is  the  preferred  software  assurance  standard  for 
safety-related  airborne  software  by  the  FAA  and  ADF.  DO-178B  pertains  to  the 
assurance  and  certification  of  all  software  requirements,  not  just  safety-related  software 
requirements,  but  does  make  specific  mention  of  the  safety-related  requirements  that  are 
imposed  on  aerospace  software  by  the  system  safety  process. 

Before  the  guidelines  of  DO-178B  can  be  applied,  a  system  safety  assessment 
process  (not  included  as  part  of  DO-178B)  is  used  to  determine  the  sources  of  any  safety- 
related  requirements  and  the  failure  condition  categories  associated  with  them.  Two 
system  safety  standards  that  may  be  used  for  this  purpose  are  MIL-STD-882C  [21]  and 
the  SAE  ARP4754  [22].  Safety-related  requirements  are  allocated  to  the  various  sub¬ 
systems  during  the  system  safety  program.  Those  requirements  that  have  been  allocated 
to  software  as  the  source,  the  detector  or  the  method  of  fault  containment,  are  allocated  a 
safety  level  for  the  most  severe  failure  condition  category  associated  with  that  component 
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of  software  from  the  following  list  in  Table  1.  For  example,  the  flight  eontrol  sub-system 
would  very  probably  warrant  a  failure  eondition  eategory  of  eatastrophie  (as  determined 
by  the  system  safety  program)  and  henee  the  software  eomponent  of  the  flight  eontrol 
subsystem  would  be  developed  as  safety  level  A  software. 


Failure  Condition  Category 

Safety  Eevel 

Catastrophic 

A 

Hazardous/Severe-Maj  or 

B 

Major 

C 

Minor 

D 

No  Effect 

E2 

Table  1.  RTCA  DO-178B  Safety  Categories  and  Software  Levels 

The  alloeated  safety  level  then  determines  the  following:  whieh  subset  of 
aetivities  must  be  eondueted,  the  degree  of  rigor  to  be  applied  to  the  aetivities,  whether 
the  assuranee  of  them  needs  to  be  eondueted  independently  from  development,  and  the 
eontrol  eategory  for  data  management  of  the  software  produets. 3 

2,  DO-178B  Objectives,  Activities,  Considerations  and  Evidence 

DO-178B  is  an  objeetive-based  standard  that  specifies  the  objeetives  for  three 
categories  of  software  lifecycle  processes;  the  three  categories  are  planning,  development 
and  integral  processes.  For  each  process,  DO-178B  defines: 

a.  Activities  to  achieve  software  life  cycle  objectives, 

b.  Design  considerations  to  support  software  lifecycle  objectives,  and 

c.  Evidence  that  demonstrates  that  software  lifecycle  objectives  have  been 
achieved. 

DO-178B  also  identifies  by  safety  level  what  control  should  be  placed  on  the  data 
items  produced  and  whether  an  objective  and  activity  should  be  conducted  by  parties 
independent  of  the  software  development. 

2  Level  E  software  does  not  require  the  applieation  of  any  DO-178B  aetivities,  eonsiderations  or 
evidenee. 

3  Use  of  the  term  ‘software  produet’  within  this  thesis  is  synonymous  with  ‘software  artifaet’  and 
eonsistent  with  the  use  in  [27]. 
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In  total,  DO-178B  lists  sixty-six  planning,  development  and  integral  objectives 
that  are  applicable  to  software  assessed  as  being  safety  level  A.  For  safety  level  D 
software,  a  subset  of  only  twenty-eight  objectives  are  required.  The  complete  set  of 
objectives  are  listed  in  Annex  A  of  this  thesis,  and  grouped  as  listed  in  Table  2.  Numbers 
in  parentheses  indicate  the  number  of  objectives  that  are  required  to  be  independently 
assessed,  rather  than  assessed  by  the  software  product  developer. 


Lifecycle  Processes 

Safety  Level 

A 

B 

C 

D 

Planning 

7 

7 

7 

2 

Development 

7 

7 

7 

7 

Verification 

40 (22) 

39(11) 

32 

8 

Configuration  Management 

6 

6 

6 

6 

Quality  Assurance 

3(3) 

3(3) 

2(2) 

2(2) 

Certification  Liaison 

3 

3 

3 

3 

Total 

66 

65 

57 

28 

Table  2.  Breakdown  of  Objectives  by  Lifecycle  Process  and  Safety  Level 


As  can  be  seen,  the  greatest  number  of  objectives  for  software  safety  levels  A,  B 
and  C  are  verification,  totaling  nearly  two-thirds  of  the  objectives  required  for  each  of 
these  safety  levels.  The  verification  activity  spans  the  range  of  software  development 
activities.  Planning,  development,  configuration  management,  quality  assurance  and 
certification  liaison  objectives  are  almost  uniformly  applied  across  safety  levels  A 
through  D. 

3.  Circumstances  for  the  Application  of  DO-178B 

In  addition  to  being  applied  to  the  development  of  original  software,  DO-178B 
may  be  applied  under  any  of  the  following  circumstances: 

a.  Software  that  was  previously  developed  and  certified  to  DO-178B  and  is 
to  be  modified  and  certified  to  the  same  safety  level  as  previously 
achieved 

b.  Software  that  was  previously  developed  for  a  different  aircraft  installation 
and  may  or  may  not  be  subject  to  modification  before  seeking  certification 
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c.  Software  that  was  previously  developed  using  a  different  development 
environment  or  developed  for  a  different  applieation  environment 

d.  Software  that  was  previously  developed  to  different  standards  or 
guidelines,  sueh  as  eommercial-off-the-shelf  (COTS)  software,  a  different 
standard,  or  to  DO-178B  but  to  a  different  safety  level. 

A  note  to  seetion  2.2.3  of  DO-178B  provides  the  following  adviee  regarding 

software  modifieation: 

The  applieant  may  want  to  eonsider  planned  funetionality  to  be  added 
during  future  developments,  as  well  as  potential  ehanges  to  system 
requirements  alloeated  to  software  that  may  result  in  a  more  severe  failure 
eondition  eategory  and  higher  software  level.  It  may  be  desirable  to 
develop  the  software  to  a  level  higher  that  that  determined  by  the  system 
safety  assessment  proeess  of  the  original  applieation,  sinee  later 
development  of  software  life  eyele  data  for  substantiating  a  higher 
software  level  applieation  may  be  diffieult. 

The  use  of  DO-178B  during  software  evolution  is  not  without  its  problems. 
Johnson  identified  literal  interpretation  of  the  eurrent  standard  and  ineorreet  applieation 
of  its  predeeessors,  DO-178A  and  DO-178,  as  areas  of  eoneern. 

Challenges  in  using  DO-178B  are  already  oeeurring.  They  inelude  the 
diseovery  that  previously  eertified  systems  didn't  neeessarily  use  earlier 
versions  of  DO-178  correetly  and  now  result  in  greater  transition  issues. 

Literal  interpretation  remains  a  problem.  [23] 

D,  MIL-STD-498  -  SOFTWARE  DEVELOPMENT  AND  DOCUMENTATION 

This  seetion  presents  a  brief  deseription  of  the  software  requirements  speeified  in 
MIL-STD-498.  MIL-STD-498  is  ehosen  as  a  representative  legaey  software  standard 
beeause  of  the  prevalenee  of  military  aerospaee  systems  still  in  operation  today  that  were 
developed  in  aeeordanee  with  (lAW)  this  standard. 

1,  Background  and  Scope  of  MIL-STD-498 

MIL-STD-498  was  developed  to  resolve  the  objections  to  MIL-STD-2167A 
Defense  System  Software  Development  and  to  merge  its  contents  with  those  of  7935A 
DoD  Automated  Information  Systems  Documentation  Standards,  thus  forming  a  single 
best-of-both  standard  that  was  consistent  with  other  Department  of  Defense  policy  and 
instructions  that  were  released  at  around  the  same  time  of  issue  of  MIL-STD-498. 


15 


One  of  the  signifieant  ehanges  to  the  requirements  in  MIL-STD-498  was  an 
attempt  to  be  independent  of  any  partieular  development  methodology.  The  standard 
provides  eonsiderable  guidanee  for  applieation  to  the  grand  design,  ineremental  design, 
and  evolutionary  design  development  methodologies  as  an  example  of  the  proposed 
flexible  use  of  the  standard.  Another  deliberate  ehange  from  earlier  standards  was  a 
reeognition  that  software  produet  data  eould  and  should  be  provided  in  alternative  forms 
to  traditional  doeuments,  in  faet  advoeating  the  use  of  natural  work  produets  rather  than 
development  of  additional  doeuments  as  evidenee  of  aetivities  and  results. 

Attempting  to  merge  weapon  system  software  and  information  system  standards 
led  to  a  doeument  that  has  a  number  of  requirements  that  have  little  or  no  relevanee  to 
airborne  software,  for  instance,  a  software  center  operating  manual.  This  situation  does 
not  present  a  problem  as  MIL-STD-498  repeatedly  advises  that  the  requirements  of  the 
standard  should  be  tailored  to  suit  the  particular  software  development  project.  Guidance 
and  instructions  for  tailoring  are  provided  in  each  of  the  general  and  detailed 
requirements  and  in  the  associated  Data  Item  Descriptions  (DID).  The  standard  also 
makes  reference  to  the  general  tailoring  guidance  in  MIL-HDBK-248  Acquisition 
Streamlining. 

2.  MIL-STD-498  Process  and  Product  Requirements 

Approximately  thirty  general  and  detailed  requirements  and  approximately 
twenty-nine  types  of  software  products  are  described  in  MIL-STD-498  which  references 
the  content  in  the  twenty-two  accompanying  DID  for  specific  information.  Development 
processes  that  are  required  by  the  standard  include: 

a.  Participation  in  system-level  activities  (requirements  through  to  testing), 

b.  Software  requirements  analysis,  design,  and  implementation, 

c.  Verification,  integration,  testing  and  corrective  action, 

d.  Configuration  and  risk  management,  metrics  analysis,  quality  assurance, 
reviews,  audits,  and 

e.  Installation  and  transition. 
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Software  development  produets  are  introdueed  and  briefly  deseribed  in  the 
standard  and  more  fully  deseribed  in  the  aeeompanying  DID.  The  22  DID  speeified  by 
MIL-STD-498  inelude; 

a.  Plans  for  the  eonduet  of  software  development  aetivities 

1 .  Software  Development  Plan 

2.  Software  Test  Plan 

3.  Software  Installation  Plan 

4.  Software  Transition  Plan 

b.  Speeifioations  of  system,  software  and  software 

1 .  System/Subsystem  Speeifieation 

2.  Interfaee  Requirements  Speeifieation 

3.  Software  Requirements  Speeifieation 

4.  Software  Produet  Speeifieation 

e.  Deseriptions  of  eoneept,  designs,  tests  and  delivered  software 

1 .  Operational  Coneept  Deseription 

2.  System/Subsystem  Design  Deseription 

3.  Interfaee  Design  Deseription 

4.  Software  Design  Deseription 

5.  Database  Design  Deseription 

6.  Software  Test  Deseription 

7.  Software  Version  Deseription 

d.  Manuals  for  users  and  support  personnel 

1 .  Software  User  Manual 

2.  Software  Center  Operator  Manual 

3.  Software  Input/Output  Manual 

4.  Computer  Operation  Manual 

5.  Computer  Programming  Manual 

6.  Firmware  Support  Manual 

e.  Software  Test  Report  to  reeord,  report  and  explain  the  results  obtained 
from  software  testing 

The  standard  stresses  the  use  of  natural  software  produets,  not  additional 


doeuments.  For  this  reason  the  produets  listed  above  are  titled  as  deseriptions,  not 
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documents,  to  remove  the  temptation  to  only  eonsider  them  as  traditional  doeuments. 
The  standard  eneourages  alternative  mediums  for  records  such  as  data  within  computer- 
aided  software  engineering  tools,  and  substitution  by  eommereial  manuals  where 
applicable. 

Whilst  MIL-STD-498  makes  the  statement  that  it  “invokes  no  other  standards,”  it 
leaves  speeifie  details  for  some  of  the  software-produet  eontent  to  related  standardization 
doeuments  sueh  as  ANSI/IEEE  Std  1008  Standard  for  Software  Elnit  Testing,  or  requires 
the  developers  to  ereate  and  follow  their  own  standards,  sueh  as  “standards  for 
representing  requirements,  design,  eode,  test  eases,  test  proeedures,  and  test  results.” 

3.  MIL-STD-498  and  Safety  Requirements 

Safety  is  regarded  as  one  of  three  speeifie  “eritieal  requirements”  along  with 
security  and  privacy.  However,  safety  eonsiderations  are  left  very  short  on  detail  about 
how  to  satisfy  them  and  make  no  distinetions  for  varying  degrees  of  safety  level.  The 
treatment  of  safety  within  MIL-STD-498  is  largely  limited  to  the  seetion  §4.2.4. 1. 

4.2.4. 1  Safety  assuranee.  The  developer  shall  identify  as  safety-eritieal 
those  CSCIs  or  portions  thereof  whose  failure  eould  lead  to  a  hazardous 
system  state  (one  that  could  result  in  unintended  death,  injury,  loss  of 
property,  or  environmental  harm).  If  there  is  such  software,  the  developer 
shall  develop  a  safety  assuranee  strategy,  ineluding  both  tests  and 
analyses,  to  assure  that  the  requirements,  design,  implementation,  and 
operating  proeedures  for  the  identified  software  minimize  or  eliminate  the 
potential  for  hazardous  eonditions.  The  strategy  shall  inelude  a  software 
safety  program,  whieh  shall  be  integrated  with  the  system  safety  program 
if  one  exists.  The  developer  shall  reeord  the  strategy  in  the  software 
development  plan,  implement  the  strategy,  and  produee  evidence,  as  part 
of  required  software  produets,  that  the  safety  assurance  strategy  has  been 
earned  out. 

§4.2.4. 1  is  a  simple  statement  of  what  is  required  without  providing  any  guidanee 
to  aehieve  it  or  issues  to  be  eonsidered  when  developing  the  required  strategy  and  plan. 
The  standard  lists  external  standards  to  supplement  the  requirements  of  MIL-STD-498, 
these  being  MIL-STD-882  System  Safety  Program  Requirements,  MIL-HDBK-272 
Safety  Design  and  Evaluation  Criteria  for  Nuelear  Weapons  Systems  and  IEEE  Std  1228 
Standard  for  Software  Safety  Plans.  Treatment  of  safety  in  the  various  MIL-STD-498 
DID  is  little  better  and  is  usually  limited  to  preeautionary  notiees,  or  speeifying  that 
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safety  requirements  should  be  singled  out  “for  special  treatment”  in  separate 
subparagraphs.  This  is  again  done  without  any  detailed  requirements,  guidance  or 
considerations  as  to  what  special  treatment  entails.  The  meager  offering  in  §3.7  of  the 
Software  Requirements  Specification  DID  (extract  included  in  Appendix  A)  is  as  detailed 
as  the  requirements  for  safety  get  in  the  standard  or  any  of  its  DID. 

4,  MIL-STD-498  Software  Product  Evaluation  and  Quality  Assurance 

Section  5.15  describes  the  evaluation  processes  for  software  products.  It 
distinguishes  between  in-process  evaluations  to  be  conducted  by  the  developer,  and 
evaluations  associated  with  formal  deliverables.  The  standard  states  requirements  for  the 
independence  of  evaluations  and  the  retention  of  records.  Appendix  D  of  MIL-STD-498 
provides  fourteen  evaluation  criteria  for  the  twenty-nine  software  products  that  it 
identifies.  The  evaluation  criteria  include  the  following  types  of  considerations: 

a.  Contains  all  the  applicable  information  of  the  relevant  DID 

b.  Meets  the  Statement  of  Work  and/or  Contract  Deliverables  Requirements 
List  if  applicable 

c.  Is  understandable  (by  the  target  audience) 

d.  Is  internally  consistent  within  a  product 

e.  Was  developed  lAW  the  software  development  plan 

f  Consistent  with  requirements  at  the  system  and  software  level 

g.  Are  feasible  to  implement 

h.  Covers  requirements,  design,  implementation  etc. 

In-process  software  Testing  (unit  testing,  unit  integration  and  testing.  Computer 
Software  Configuration  Item  (CSCI)/Hardware  Configuration  Item  integration  and 
testing)  do  not  receive  the  same  treatment  as  formal  qualification  of  CSCI  and  system 
testing.  In-process  testing  does  not  require  testing  personnel  independence  from 
development  personnel  and  does  not  provide  any  guidance  for  evaluation  criteria  beyond 
the  term  ‘adequate.’  The  ADF  Airworthiness  Design  Requirements  Manual  has  the 
following  statement  regarding  the  adequacy  of  MIL-STD-498. 

While  many  of  the  objectives  under  RTCA/DO-I78B  have  placeholders  in 

MIL-STD-498,  there  are  no  criteria  that  can  be  used  to  assess  the 

adequacy  of  completion  of  the  activity.  For  example,  there  is  a 

requirement  for  unit  and  integration  testing,  but  there  are  no  criteria  that 
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define  when  testing  can  be  considered  complete.  Therefore  MIL-STD-498, 
in  isolation,  does  not  provide  an  adequate  basis  for  software  assurance 
and,  by  itself,  is  not  recognized  by  the  TAR  as  a  software  assurance 
standard.  [14] 

Software  Quality  Assurance  of  process  and  product  is  covered  in  section  5.16  of 
MIL-STD-498  (extract  included  in  Appendix  A).  The  purpose  of  this  process  is  to  ensure 
that  activities  are  carried  out;  that  the  respective  software  products  are  produced  and 
evaluated;  and  that  identified  problems  are  recorded,  analyzed  and  corrected  or  justified. 
The  standard  also  requires  that,  quality  assurance  activities  be  conducted  by  personnel 
that  are  independent  of  the  development  and  product  evaluation  activities,  and  that 
quality  assurance  records  be  generated  and  retained. 
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III.  SOFTWARE  EVOLUTION,  CERTIFICATION  AND  THE 
RELATIONSHIP  WITH  SOFTWARE  STANDARDS 


A,  SOFTWARE  EVOLUTION  AS  REENGINEERING 

A  useful  representation  of  the  steps  involved  during  maintenanee  or 
modernization  has  been  proposed  by  Kazman,  Woods  and  Carriere  [24],  The  model  in 
Figure  1  shows  the  multiple  paths  that  software  evolution  may  take  from  legaey  eode  to 
new  code. 


□ssign 

pattftms 


P^iogram 

plan;: 


ArGhitetti_*ie 

r<=pr!?se<^ation 


Function- I&^;bI 
mprft5enr.3ilKjn 


CodB  at''L>ctur'e 
representinpn 


rtCfiesemarkOrt 


Fhirrborv-lft^iifti 

reprce^amaijaF'i 


Code 

ryprasunLar-oii 


sourrift 


SO'jrcy  iBKt 
rdpr€SBnra:ion 


'_aga:>- 

^Quroe 


Figure  1.  Model  for  Software  Reengineering  (From:  [1]) 


The  horseshoe  model  reveals  a  vertical  path  up  the  left  side  for  understanding  the 

software  by  reconstruction  from  less  to  more  abstract  software  products;  lateral  paths 

across  the  model  for  code,  functional  and  architectural  transformations;  and  a  vertical 

path  down  the  right  side  for  refinement  from  software  abstractions  to  code.  It  is  not 

always  necessary  or  desirable  to  travel  the  full  path  around  the  outside  of  the  horseshoe. 

This  would  only  be  necessary  if  a  major  modernization  effort  was  being  undertaken  and 
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the  only  software  product  available  from  the  legacy  software  development  is  the  code, 
thus  forcing  a  reengineering  project.  At  the  other  end  of  the  software  evolution  spectrum 
is  minor  modification.  This  may  be  achieved  by  a  direct  code  transformation  without 
attempting  to  reconstruct  design  and  architecture. 

In  all  cases  of  software  evolution,  the  developer  makes  a  practical  choice  of 
starting  point  on  the  left,  reconstructs  only  as  much  as  is  necessary  (if  any),  then  makes 
the  transformation  at  that  level,  finally  proceeding  through  the  refinement  process  to  the 
new  code  (possibly  only  a  partial  refinement  intended  to  simply  incorporate  new 
requirements). 

B,  SOFTWARE  PRODUCT  VERIFICATION 

There  is  debate  within  the  software  engineering  community  over  the  merits  or 
otherwise  of  inferring  the  quality  of  fielded  software  code  from  the  quality  of  the  people, 
processes,  and  tools  that  an  organization  uses  to  develop  software  products.  It  is 
generally  accepted  that  whilst  quality  processes  are  a  necessary  enabler  to  produce 
quality  code,  processes  alone  are  not  a  guarantee  of  quality  code.  It  is  for  this  reason,  that 
the  bulk  of  software  processes  for  safety-critical  software  are  in  fact  the  software  product 
verification  processes  that  include  a  wide  range  of  activities  from  requirements 
verification  through  unit  testing  to  final  qualification  testing.  Figure  2  [25]  shows  a 
taxonomy  of  the  major  types  of  verification  activities  that  must  be  conducted  on  various 
software  products.  The  testing  branch  of  the  verification  techniques  can  be  broken  down 
further  into  the  following  sub-types: 

1 .  Requirements-based  testing  that  addresses  the  system  and  software 
functional  and  non-functional  requirements 

2.  Function-based  testing  that  addresses  the  software  design  functions 

3.  Structure-based  testing  that  addresses  the  software  implementation 

4.  Data-based  testing  that  addresses  different  categories  of  data  inputs,  e.g. 
random  inputs,  equivalent  partitions,  normal  inputs,  abnormal  inputs  and 
boundary  values 

5.  State-based  testing  for  state-based  software 

6.  Probability-based  testing  to  assess  software  reliability 

7.  Fault- injection  testing  that  uses  prior  experience  to  target  likely  sources  of 
error,  tests  fault-tolerance  performance,  or  tests  the  test. 
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Figure  2  Verification  Techniques  (From:  [25]) 

Given  the  practical  impossibility  of  performing  exhaustive  testing,  an  important 
consideration  for  the  verification  of  safety-critical  software  is  the  measure  of  coverage 
that  is  provided  by  the  verification  effort.  The  quantity  and  type  of  verification  that  is 
performed  on  software  will  determine  the  level  of  assurance  that  can  be  ascribed  to  the 
fielded  product. 

C.  SOFTWARE  EVOLUTION  AND  CERTIFICATION 

Recertification  must  be  sought  when  significant  hardware  or  software  changes  are 
made  to  an  aircraft  design.  MIL-HDBK-514  Operational  Safety,  Suitability,  & 
Effectiveness  for  the  Aeronautical  Enterprise  [26]  provide  the  following  instances  that 
warrant  recertification  of  airworthiness: 

1 .  Changes  that  affect  propulsion/drive  system  operation  (including  software) 

2.  Significant  software  revisions 

3.  Modification  to  weapons  release/firing  system,  including  stores 
management  system  and  associated  weapons  system  software 

1.  Using  Deductive  Logical  for  Software  Certification 

Software  certification  is  based  on  a  conclusion  that  software  meets  its  quality 
specifications.  In  this  paper,  the  quality  specification  of  interest  is  safety;  there  are  of 
course  other  quality  objectives  that  exist,  for  instance  security,  which  have  their  own 
certifications  requirements.  Accepting  that  the  absolute  definition  of  safety  is  not 
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achievable  for  present-day  complex  systems,  the  practical  conclusion  that  is  drawn  is  that 
a  software  produet  meets  alH  of  the  software  assurance  objectives  of  an  aeceptable 
software  assurance  standard.  A  conclusion  should  only  be  aecepted  as  valid  when  it 
flows  naturally  from  the  premises  upon  whieh  it  is  founded. 

The  conclusion  about  software  eertification  may  be  either  positive  (accepted)  or 
negative  (rejected)  and  in  either  ease  is  based  on  one  of  the  following  implieations: 

a.  If  all  of  the  software  assurance  objectives  are  satisfied,  then  the  software 
is  certifiable  lAW  the  aecepted  software  assurance  standard,  or 

b.  If  any  of  the  software  assurance  objectives  are  not  satisfied,  then  the 
software  is  not  certifiable  lAW  the  accepted  software  assurance  standard. 

What  remains  to  be  determined  for  this  basis  of  a  eonelusion  is  a  technique  to 
establish  which  of  the  above  propositions  is  true.  The  following  is  a  two-part  proposal  to 
achieve  certification  of  evolved  software: 

a.  Acceptance  or  rejection  of  evidence  that  was  produeed  during  the  prior 
development  of  the  software  that  is  to  be  modified,  and 

b.  Aceeptance  or  rejeetion  of  evidence  that  is  newly  produced  during  the 
development  of  the  evolved  software. 

It  is  understandable  that  not  all  of  the  evidence  presented  from  prior  development 
will  be  aceepted  as  meeting  the  objeetives  of  a  contemporary  software  assurance  standard 
and  in  some  cases  will  not  even  be  applicable.  In  the  cases  where  the  prior  evidenee  is 
either  inadequate  or  non-existent,  new  evidenee  will  be  required.  New  evidenee  must 
also  be  produeed  for  the  parts  of  the  software  development  that  are  unique  to  the 
modification  of  the  software.  The  focus  of  this  thesis  is  a  framework  for  determining  the 
adequacy  of  the  existing  software  produets  in  support  of  airworthiness  eertification  of 
evolving  software. 

2.  Establishing  the  Safety  Level  Assessment 

One  of  the  early  steps  to  be  undertaken  when  commencing  modification  of  safety- 
critical  software  is  to  either  establish  or  revise  the  safety  level  for  the  modified  software. 
A  valid  assessment  of  safety  level  is  important  as  it  determines  the  activities  that  need  to 

4  Excluding  waivers  which  are  left  as  an  issue  for  certification  authorities  that  deal  with  them  on  a 
case-by-case  basis. 
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be  conducted  and  the  evidence  that  will  need  to  be  presented  to  an  airworthiness  authority 
to  achieve  certification.  Assessing  the  safety  level  can  be  achieved  in  the  first  instance, 
by  examining  the  existing  system/software  safety  program  outputs  from  the  previous 
development.  When  doing  so,  the  assumptions  made  and  analysis  conducted  for  the 
previous  development  must  be  reviewed  in  the  light  of  the  proposed  modifications,  and 
then  hazard  identification  and  analysis  must  be  conducted  for  the  new  requirements  that 
are  proposed.  If  a  system/software  safety  program  was  not  conducted  for  the  previous 
development,  or  the  information  is  insufficient  or  unavailable  for  any  reason,  a  complete 
hazard  identification  and  analysis  will  need  to  be  conducted.  The  final  outcome  of  this 
effort  is  the  assessment  of  the  failure  condition  category  and  requisite  safety  level  for  the 
modified  software. 

3,  Carrying  Out  Software  Evolution 

Once  the  software  level  has  been  determined,  development  should  proceed  in  a 
manner  that  facilitates  the  granting  of  an  airworthiness  certification.  What  this  means  is 
that  sufficient  evidence  must  be  produced  throughout  the  life-cycle  that  demonstrates  the 
successful  completion  of  the  software  engineering  activities  that  address  the  objectives  of 
software  assurance. 

4.  Achieving  Airworthiness  Certification 

The  final  step  to  achieving  airworthiness  certification  is  the  collation  of  the 
evidence  that  supports  certification.  Two  well  known  formats  for  this  are  the  software 
accomplishment  summary  described  in  DO-178B  or  the  safety  case  described  in  DEF 
STAN  00-55. 

D,  SOFTWARE  EVOLUTION  AND  STANDARDS 

The  long  time-frames  over  which  aerospace  systems  are  developed  and  then 
continue  to  evolve  expose  them  to  ongoing  changes  in  software  engineering.  This 
evolution  of  the  discipline  of  software  engineering  is  manifest  in  the  new  computing 
technologies,  and  the  design,  implementation  and  verification  techniques  that  are 
developed  to  cope  with  the  increasing  complexity  of  modern  software-intensive  systems. 
Another  source  of  software  engineering  evolution  that  directly  affects  the  standards 
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domain  is  military  acquisition  reforms  to  reduce  the  number  of  military  standards;  efforts 
in  this  area  aim  to  cancel  rewrite  or  replace  military  standards  with  acceptable 
commercial-equivalent  standards. 

Given  the  cost  to  develop  safety-critical  software  for  aerospace  applications,  a 
reasonable  expectation  during  software  evolution  is  to  be  able  to  reuse  the  products  that 
were  produced  during  the  previous  software  development.  The  challenge  to  achieving 
cost-  and  time-effective  software  reuse  and  the  subsequent  certification  of  the  system 
containing  the  reused  software  is  to  identify  the  relationship  between  the  activities  and 
outputs  of  the  previous  development,  and  the  activities  and  outputs  required  for  the 
modification.  This  thesis  presents  a  model  for  identifying  the  necessary  relationships  to 
facilitate  software  reuse  in  support  of  the  certification  of  evolving  safety-critical  software. 

The  first  version  of  software,  version  X  in  Figure  3,  has  a  relationship  Ri,  to 
standard  Si,  that  meets  or  exceeds  the  requirements  for  certification  lAW  standard  Si  at 
the  time  of  development  of  X.  Software  products  must  comply  with  multiple  standards  if 
no  one  standard  provides  all  of  the  requirements  needed  for  a  software  development 
project.  Relationship  Ri  explicitly  represents  just  the  one  standard  of  interest;  for  the 
purposes  of  this  thesis  the  type  of  standard  of  interest  is  that  of  software  assurance. 


The  final  set  of  software  products  that  is  produced  during  software  development 
provides  a  complete  definition  of  software  version  X  in  Figure  3.  Flowever,  it  is  only  a 
subset  of  this  final  set  that  is  necessary  to  satisfy  the  requirements  of  standard  Si.  The 
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compliment  of  this  subset  contains  the  additional  products  sometimes  regarded  as  internal 
to  the  development  organization  but  which  are  necessary  enablers  for  the  activities  that 
complete  the  software  development. 

Version  X  of  the  software  also  has  a  relationship  R2,  to  the  contemporary  standard 
that  is  now  preferred,  but  either  was  not  available,  or  was  not  applied  at  the  time  of 
previous  development.  This  relationship  exists  from  the  moment  that  both  standard  S2 
and  version  X  of  the  software  both  exist,  but  may  be  of  little  praetical  interest  until  the 
requirements  of  standard  S2  are  invoked  for  version  X  of  the  software. 5 

Version  X'  of  the  software  has  a  relationship  to  version  X  (R3),  which  is  the  result 
of  modification  due  to  maintenance  or  modernization.  Some  of  the  reasons  for  this 
modernization  may  be  the  addition  of  new  features,  removal  of  existing  features, 
retargeting  the  software  to  new  hardware  or  revamping  of  the  interface.  Version  X'  of  the 
software  also  has  a  relationship  R4,  to  standard  S2,  which  is  the  foeus  of  this  thesis.  In 
order  to  achieve  certification  of  version  X',  relationship  R4  must  satisfy  the  requirements 
of  standard  S2  to  the  satisfaction  of  the  certification  authority.  Versions  X"  and  X'"  and 
relationships  R5  through  Rg  are  further  iterations  of  the  same  principle  for  subsequent 
modifications  of  the  software.  This  model  can  also  be  applied  to  the  introduction  of  a 
third  software  assurance  standard,  in  which  case  second  standard  (formerly  S2)  would  be 
represented  by  Si,  and  the  third  standard  represented  by  S2.  Furthermore,  the  standards 
represented  by  Si  and  S2  could  be  a  different  version  of  the  old  standard,  for  instance 
DO-178B  and  the  pending  DO-178C.  Using  this  construction  of  the  relationships,  the 
evolution  of  the  software  product  is  represented  by  Equation  1 : 

X"  =XLJ(AXj 

i'=l 

Equation  1 .  Software  Evolution 


5  Relationship  R2  may  never  be  invoked  and  in  faet  is  unlikely  to  be  invoked  if  version  X  does  not 
undergo  any  further  ehanges. 
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AXj  represents  the  modification  introduced  at  each  iteration  of  the  software 


product,  such  as 


X"'  =  X'VAX, 

=  XVAX^\JAX, 

=  X\JAX,\JAX^\JAX, 

Further  development  of  an  expression  for  software  evolution  in  terms  of  the 
individual  products  to  compose  any  given  version  requires  an  expression  for  software 
engineering  that  produces  a  single  software  version.  A  description  of  such  an  expression 
follows  next. 

E.  ABSTRACT  ALGEBRA  AND  MORPHISMS 

Before  applying  the  concepts  of  abstract  algebra  to  software  engineering,  we 
briefly  review  the  theory  of  abstract  algebra  in  its  familiar  mathematical  domain.  In 
mathematics,  all  algebras  are  defined  by  a  set  of  symbols  A,  and  the  set  of  operations  fl , 
that  can  be  applied  to  the  elements  of  A.  Two  algebras  are  said  to  be  similar  if  they  have 
the  same  number  of  operations  for  each  arity.  A  purely  hypothetical  case  would  be  two 
algebras  are  considered  to  be  similar  if  they  both  have  three  unary  operations,  seven 
binary  operations  and  two  ternary  operations.  Furthermore,  two  algebras  are  said  to  be 
homomorphic  if  there  is  a  function  that  provides  a  one-to-one  correspondence  between 
every  element  of  one  symbol  and  operation  set  [A^QJ  and  another  symbol  and 

operation  set  [Aj  ,^2  ]  • 

If  ^/i((y(al,a2,...,a„))  =  ^y'(^(al),^/i(a2),...,^/i(a„)) 
for  every  ry  e  Q, 
for  every  corresponding  co'  e  fl', 
for  every  a.  e  A 

where  n  is  defined  by  the  arity  of  co 
Then  is  a  homomorphism  from  [A,Q]  to  [A',Q'] 

Definition  of  Homomorphism 


28 


To  illustrate  this  with  a  common  example,  consider  the  homomorphic  function  of 
logarithm  that  provides  such  a  correspondence  between  multiplication  and  addition  and 
the  set  of  symbols  (numbers)  that  are  valid  for  logarithms.  One  algebra  R,  is  defined  by 
the  symbol  set  of  all  positive  real  numbers  and  the  single  binary  operation  of 

multiplication,  that  isR  =  .  A  second  algebra  L,  is  defined  by  the  symbol  set  of 

all  real  numbers  (negative  and  positive)  and  the  single  binary  operation  of  addition,  that 
isT  =  .  The  reader  might  be  tempted  to  propose  that  there  are  many  other 

operations  that  could  be  performed  on  the  elements  and  M. ,  but  those  operations 
would  be  outside  the  definition  presented  here,  and  constitute  a  different  algebra  than 
either  R  or  L.  Note  also,  that  there  is  no  requirement  for  the  symbol  sets  Ai  and  A2  to  be 
equivalent.  The  two  algebras  R  and  L  in  this  example  are  similar  by  virtue  of  each 
having  only  one  binary  operation.  Furthermore,  algebras  R  and  L  are  said  to  be 
homomorphic  by  reason  of  the  function  ^zi(x)  =  log  (x) .  For  example: 

Using  ^/i(o(ai,a2,...,aJ)  =  ®'(^/i(ai),^(a2),...,^(a„)) 
substituting  (f)  =  Log 
Tog  («! ,  02 ))  =  Q)'[Log  (oi ) ,  Log  (02 )) 
substituting  ®  =  x,  and  co'  =  + 

Log  (x  (oi ,  02 ))  =  +  (Tog  (oi ) ,  Tog  (02 )) 
hence 

Tog (oj  X  02 )  =  Log (oj )  +  Log (02 ) 

Equation  2.  Homomorphism  Example  -  Eogarithms 

Another  example  of  a  homomorphic  function  is  the  Eaplace  Transform  that 
permits  convolution  in  the  linear  time  domain  to  be  represented  as  multiplication  in  the 
frequency  domain. 

F.  ABSTRACT  ALGEBRA  AND  SOFTWARE  DEVELOPMENT 

The  relationships  in  Figure  3  are  composed  of  the  products  (symbol  set)  that  are 
used  and  produced  during  the  activities  (operations)  that  are  conducted  during  software 
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evolution.  Before  applying  the  mathematical  concept  of  homomorphism  to  software 
evolution  and  standards  interoperability,  we  first  specify  the  sets  A  and  Q  as  follows; 

1 .  A  represents  the  set  of  all  possible  software  products  ( «^ )  both  used  and 
created  during  software  development,  and 

2.  fl  represents  the  set  of  all  possible  activities  ( )  that  are  conducted  on 
the  software  products  during  software  development. 

Some  obvious  examples  of  the  products  ,  in  set  A  are  the  Plans,  Requirements, 

Infrastructure,  Design,  Source  Code,  Executable  Code,  Verification  results.  Section  1.5 
of  the  IEEE  Standard  for  Software  Reviews  [27]  has  a  list  of  thirty-seven  ‘software 
products’  with  the  inclusion  of  such  items  as  ‘anomaly  reports,’  ‘build  procedures,’ 
‘installation  procedures,’  and  ‘walkthrough  reports’  in  addition  to  the  previously 
mentioned  products.  Examples  of  the  activities  ,  in  set  Q  are  Planning,  Development, 

Verification,  Configuration  Management,  Quality  Assurance,  Certification  etc.  for  the 
various  stages  of  software  development. 

Using  the  algebraic  representation  for  software  development  we  can  now  write; 

where  conducting  activity  , 

on  an  appropriate  subset  , 

that  has  been  produced  prior  to  activity  , 

produces  a  new  software  artifact  . 

Equation  3.  Single  Product  Development 

Equation  4  provides  an  example  of  this  algebraic  description  as  the  development 
of  code; 


Software  Coding 


^  Development  Plan  ^ 
Coding  Standards 


Code 


Equation  4. 


Development  Tools 
Design 

Single  Product  Development  -  Code  Example 


30 


Each  of  the  products  used  in  the  software  coding  activity  (which  in  this  example 
is  a  quaternary  operation)  are  themselves  products  of  earlier  software  development 
activities,  that  is,  planning,  defining  standards,  choosing  and  qualifying  tools  and 
developing  the  software  design.  Development  tools  also  encompass  configuration 
management  and  traceability  systems  in  addition  to  the  obvious  tools  such  as  text  editors 
and  compilers.  Design  includes  the  software  architecture  and  design-level  software 
requirements.  6 

While  the  source  and  object  code  are  being  produced  by  the  software  coding 
activity,  other  important  products  are  also  being  generated.  These  additional  products 
include  updates  to  traceability  information  and  configuration  management  data  in 
addition  to  the  creation  of  feedback  for  the  design  and  requirements  activities  in  case  any 
errors  (e.g.,  conflicting  requirements)  are  found  in  them  as  a  consequence  of  carrying  out 
the  software  coding  activity.  Including  these  products  into  the  representation  expands  the 
previous  description  of  software  development  to  that  of  Equation  5: 

where  conducting  activity  , 

on  an  appropriate  subset  , 

that  has  been  produced  prior  to  activity  , 

produces  the  new  subset  of  the  software  artifacts  , 

such  that  (5^  c:  Q). 

Equation  5.  Single  Activity 

Equation  5  specifies  that  the  set  of  products  that  are  produced  (Cs)  by  an  activity 
is  a  proper  superset  of  the  products  used  (Bs)  during  the  activity.  This  would  always  be 
the  case,  even  in  situations  were  some  portion  of  the  requirements,  design  or 
implementation  is  removed.  At  the  very  least,  the  removed  product  should  remain  as  part 
of  the  development  history  for  the  software. 

6  As  distinct  from  system  or  high-level  software  requirements. 
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The  source  code  development  example  presented  in  Equation  4  is  expanded  to 
have  the  following  relationships  that  demonstrate  the  production  of  multiple  software 
products  from  a  single  software  development  activity: 


Software  Coding 


^  Development  Plan  ^ 
Coding  Standards 
Development  Tools 
Design 


Code 

Traceability  Data 
CM  Data 
Trouble  Reports 


Equation  6.  Single  Activity  -  Coding  Example 


The  last  step  in  the  algebraic  representation  of  software  development  is  to 
represent  the  composition  of  all  the  development  activities  that  together  produce  the 
completed  software  package. 


m 

5=1 

where  the  union  of  m  activities  co^ , 
on  the  appropriate  subsets  of  artifacts  , 
produces  the  final  set  of  artifacts  Aj. 

Equation  7.  Software  Development 

The  last  step  in  the  algebraic  representation  of  software  development  is  to 
represent  the  composition  of  all  the  development  activities  that  together  produce  the 
completed  software  package. 

G.  ABSTRACT  ALGEBRA  AND  SOFTWARE  EVOLUTION 

Combining  the  representations  in  Equation  1  and  Equation  7  provides  the 
composite  expression  in  Equation  8  for  the  total  output  of  software  development  as  a 
product  evolves. 
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For  X"=x(][^X,) 

s=\ 

where  X  =  U  (-6^ ) 


and  AX  =  U(»,  (-S,) 
then  X"  =  U  (o,  |j[  U  (o,  [B, ) 


t=\ 


i=i 


Equation  8.  Software  Evolution 


It  is  desirable  from  a  eost-benefit  perspeetive  that  some  of  the  products  that  were 
produced  during  previous  software  development  be  reused  to  achieve  certification  during 
software  evolution.  In  terms  of  the  model  presented  in  this  thesis,  this  amounts  to  how 
much  of  the  software  product  package  X"  satisfies  the  relationship  R2(n+i).  The  greater 
the  amount  of  previously  developed  products  that  can  be  reused  is  clearly  an  important 
business  objective,  but  this  must  be  achieved  within  the  bounds  of  an  acceptable  level  of 
software  assurance  to  meet  airworthiness  requirements. 

In  terms  of  the  first  iteration  of  software  evolution,  how  much  of  the  development 
for  version  X ,  is  available  as  reused  software  product  from  the  development  of  version 
X,  can  be  legitimately  used  to  satisfy  the  relationship  R4?  A  promising  answer  to  this 
problem  lies  in  how  much  of  the  development  for  version  X,  that  was  used  to  satisfy 
relationship  Ri  could  have  been  used  to  satisfy  the  relationship  R2?  The  next  section  of 
this  chapter  examines  this  question. 

H.  APPLICATION  OF  MORPHISM  TO  SOFTWARE  EVOLUTION 

The  mathematical  concept  of  homomorphism  was  introduced  in  §IILE  in  order  to 
use  it  now  as  the  basis  for  describing  the  correspondence  of  the  products  that  constitute 
the  relationships  Ri  and  R2.  A  function  that  transforms  the  products  developed  for 
version  X  and  satisfies  relationship  Ri,  to  products  that  satisfy  relationship  R2  can  be 
used  to  identify  the  amount  of  reuse  of  existing  software  product  that  is  available  for 
certification  of  the  evolved  software.  Unfortunately,  the  definition  of  homomorphism  has 
a  single  function  that  provides  the  mapping  between  two  sets  of  similar  algebra.  It  would 
indeed  be  nice  if  a  single  function  existed  to  map  all  the  products  satisfying  relationship 
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Ri  to  also  satisfy  relationship  R2.  It  would  be  perfeet  if  sueh  a  funetion  was  an  identity 
funetion  and  hence  no  additional  operations  or  activities  were  required  to  achieve  the 
desired  correspondence.  This  is  highly  unlikely  given  that  standard  S2  either  did  not  exist 
or  was  not  chosen  as  the  basis  for  certification  of  the  previous  software  development. 
Three  likely  scenarios  for  morphims  to  the  products  that  satisfy  relationship  Ri  to  make 
them  acceptable  for  relationship  R2  are; 

1 .  Some  of  the  existing  products  will  be  able  to  be  mapped  using  an  identity 
function,  in  which  case  no  additional  activity  is  required  for  them  to  be 
considered  as  meeting  the  requirements  of  S2, 

<l>{cc,)^cc, 

Equation  9.  Identity  Morphism 

2.  Some  of  the  existing  products  will  be  able  to  be  mapped  using  a  partial 
function,  in  which  case  some  rework  or  additional  activity  is  required  for 
these  to  be  used  to  meet  the  requirements  of  S2, 

^  da^ 

Equation  10.  Partial  Morphism 

3.  Some  of  the  existing  products  will  not  be  able  to  be  mapped  using  any 
function;  or  rather  these  products  will  be  subject  to  a  null  function  in  the 
transformation.  In  these  cases,  completely  new  work  and  software 
products  will  be  required  to  satisfy  the  requirements  of  relationship  R2. 

«»(«,)  ^0 

Equation  1 1 .  Null  Morphism 

Achieving  effective  reuse  during  airworthiness  certification  is  a  matter  of 
maximizing  the  amount  of  products  that  can  be  found  in  the  first  two  of  these  groups. 
Ensuring  adequate  airworthiness  is  a  matter  of  ensuring  that  products  that  are  in  the  last 
two  groups  are  not  mistakenly  included  in  the  first  group. 

This  concept  of  morphism  is  applied  to  the  software  coding  activity  in  the 

following  sections  as  an  example  for  code  generation.  The  next  section  has  an 

underlying  premise  that  certification-related  aspects  to  code  generation  that  are  important 
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for  safety-critical  software  are  ensuring  that  software  code  and  requirements  are  fully 
traceable  in  both  directions,  and  that  code  has  been  verified  as  correct  and  complete. 
These  aspects  are  described  in  the  following  paragraphs  and  presented  in  the 

^  (Q  )  algebraic  notation. 

1.  Previous  Software  Development 

The  activity  of  software  coding  has  plans,  standards,  tools  and  design  as  inputs  to 
this  process.  The  chief  product  that  is  produced  by  the  coding  process  (at  least  with 
respect  to  an  executable  product)  is  code.  Other  products  that  are  produced  by  the 
software  coding  activity  are  traceability  data,  configuration  management  data  and  trouble 
reports.  These  are  not  necessary  to  produce  an  executable  product  and  may  be  regarded 
by  the  code-centric  developer  as  by-products  of  producing  executable  code,  but  they  are 
essential  to  achieving  the  software  assurance  necessary  for  a  certifiable  product. 
Equation  6  is  presented  again  below  for  the  sake  of  completeness  in  this  section. 


Software  Coding ^ 


Development  Plan^ 
Coding  Standards 
Development  Tools 
Design^ 


Code^ 

Traceability  Data^ 
CM  Data^ 

Trouble  Reports^ 


Equation  6.  Single  Activity  -  Coding  Example 


Eor  safety-critical  software,  it  is  necessary  to  verify  that  all  low-level 
requirements  have  been  met  and  to  also  ensure  that  there  is  no  additional  code  that  does 
not  have  a  traceable  link  back  to  requirements.  Establishing  traceability  between  code 
and  design  (i.e.,  low-level  requirements)  is  a  process  that  has  as  inputs  to  the  process; 
code,  design  for  that  code,  and  development  tools  that  permit  traceability  to  be  recorded. 
The  output  traceability  data  is  added  to  the  traceability  database  of  the  software  project. 


Code  Traceability 


Design^ 

Code^ 

^Development  Toolsy 
Equation  12.  Code  Traceability 


Traceability  Data^ 
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The  purpose  of  software  testing  is  to  ensure  that  the  software  design  has  been 
implemented  and  does  not  contain  errors.  The  inputs  to  this  process  are  the  code  itself 
and  the  verification  tools  that  enable  testing.  (For  the  sake  of  simplicity  in  this  example, 
other  inputs  such  as  test  scripts  are  considered  to  be  part  of  the  verification  tools.)  The 
outputs  from  the  testing  process  are  the  obvious  test  results,  but  also  include  traceability 
and  configuration  management  data  and  trouble  reports. 


( 


Software  Testing ^ 


Code^ 


Test  Infrastructure 
Test  Plans,  Cases  and  Procedures^ 
yTest  Input  Data^ 


Test  Logs  and  Results^ 
Traceability  Data^ 

CM  Data^ 

Trouble  Reports^ 


Equation  13.  Software  Testing 


This  completes  the  description  of  the  previous  development  of  software  products 
associated  with  the  software  coding  and  the  additional  activities  necessary  for 
certification  of  this  small  element  of  software  development.  The  next  section  describes 
the  software  coding  and  related  activities  for  modification  of  the  legacy  software. 

2.  Software  Modification 

Plans,  standards,  tools  and  design  are  once  again  the  inputs  to  the  software  coding 
activity.  However  some  of  the  activities  undertaken  and  artifacts  used  during 
modification  may  not  be  the  same  as  those  for  the  legacy  software  development.  The 
plans  and  design  used  for  software  modification  will  likely  be  different  to  that  used  for 
the  previous  development,  while  the  coding  standards  and  development  tools  may  or  may 
not  be  the  same  as  previously  used.  In  this  example,  it  is  assumed  that  the  coding 
standards  and  development  tools  are  in  fact  the  same  as  that  used  for  the  previous 
development.  Furthermore,  no  distinction  is  drawn  between  the  deliberate  reuse  of 
standards  and  tools  and  the  unplanned  reuse  of  the  same  that  would  be  characterized  as 
salvage  rather  than  reuse.  New  trouble  reports,  updated  traceability  data  and 
configuration  data  will  again  be  generated  in  addition  to  the  code  as  a  consequence  of  the 
modification. 
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As  the  software  coding  activity  is  being  conducted  for  the  purposes  of 
maintenance  or  modernization  and  not  replacement,  the  actual  code  produced  during  the 
Software  CodingM  activity  is  likely  to  be  just  a  fraction  of  the  total  code  that  defines  the 
final  product.  These  considerations  are  expressly  represented  in  Equation  14.  In  addition 
to  the  modified  code,  the  code  that  is  unchanged  from  the  previous  development  must 
retain  its  traceability,  correctness  and  completeness  during  the  modification.  The  final 
modified  code  is  described  by  Equation  15. 


Software  Coding 


M 


Development  Planj^ 
Coding  Standards 
Development  Tools 
Design^ 


Code 


M 


Traceability  Data,^ 
CM  Data^ 

Trouble  ReportSi^j 


Equation  14.  Software  Coding  -  Modification 


Code  =  (Codcp )  U  (Code^ ) 
Equation  15.  Software  Code  Evolution 


Carrying  out  the  activity  of  code  traceability  is  also  going  to  be  a  process  that 
considers  both  modified  and  unaffected  software  products.  In  the  same  manner  as  for 
code  generation,  traceability  data  will  be  described  by  the  union  of  outputs  from  both  pre- 
and  post-  modification  traceability  products  described  by  Equation  16  and  Equation  17. 


f 

Code  Traceability,^ 

V 

Equation  16. 


Traceability  Data 


Design^ 

Code^i 

Development  Tools^ 

Code  Traceability  -  Modification 


M 


Traceability  Data  =  (Traceability  Data^ )  U  (Traceability  Data,^ ) 
Equation  17.  Software  Traceability  Evolution 
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The  same  rationale  applies  to  the  software  testing  activity  conducted  for  the  final 
modified  software  and  the  description  of  configuration  management  data,  trouble  reports 
and  test  results  that  are  produced  by  the  activity.  These  are  represented  by  Equation  18 
through  Equation  21 . 


Software  Testing,^ 


Code 


M 


Test  Infrastructure 

Test  Plans,  Cases  and  Procedures,^ 

Test  Input  Data,^ 


Test  Logs  and  Results,^ 
Traceability  Data,^ 

CM  Data,^ 

Trouble  Reports,^ 


Equation  18.  Software  Testing  -  Modification 

Test  Logs  and  Results  =  (Test  Logs  and  Results^ )  U  (Test  Logs  and  Results^ ) 
Equation  19.  Software  Test  Results  Evolution 


CM  Data  =  (CM  Data^ )  U  (CM  Data,^ ) 

Equation  20.  Software  Configuration  Management  (CM)  Data  Evolution 


Trouble  Reports  =  (Trouble  Reports^ )  U  (Trouble  Reports,^ ) 

Equation  21 .  Software  Trouble  Reports  Evolution 

3.  Software  Product  Morphism 

The  previous  two  sections  described  a  portion  of  the  software  development 
(software  coding  and  related  activities);  first  for  previous  software  coding,  and  then  for 
the  subsequent  modification  of  the  code.  This  section  addresses  which  of  the  software 
products  from  this  total  effort  of  previous  development  and  modification  might  be  used  to 
satisfy  software  assurance  certification  of  the  final  modified  software.  Some  of  the 
products  from  the  previous  development  may  be  satisfactory,  even  if  the  currently  desired 
standard  was  not  met  for  the  previous  development,  ft  is  likely  however  that  at  least 
some  of  the  software  product  will  not  be  acceptable  for  the  current  certification  effort, 
even  if  the  software  assurance  standard  has  not  changed,  this  being  the  more  likely 
scenario  if  a  new  software  assurance  standard  has  been  imposed  upon  the  modification. 
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a.  Code  Traceability 

We  firstly  deal  with  traeeability  between  design  and  eode,  whieh  we  have 
asserted  is  one  of  the  eertifieation  requirements  assoeiated  with  the  software  eoding 
activity  in  order  to  meet  a  software  assurance  standard.  Equation  17  shows  the  two 
components  that  comprise  the  full  code  traceability  for  the  final  modified  code;  existing 
and  modification  traceability  data.  Acceptance  that  the  software  modifications  were 
carried  out  with  the  preplanned  intention  to  meet  a  new  software  assurance  standard, 
leads  to  a  reasonable  expectation  that  the  traceability  data  generated  for  the  modified 
segments  of  code  will  be  acceptable.  Traceability  between  unmodified  code  and  design 
may  not  be  properly  established  after  modification,  especially  if  the  traceability  prior  to 
modification  is  questionable;  this  element  of  traceability  is  explored  further  in  the 
following  paragraphs. 

Table  3  shows  a  simplified  example  that  demonstrates  some  of  the 
variations  to  code  traceability  as  a  consequence  of  modifications  to  code.  Columns  one 
and  two  represent  traceability  between  design  elements  and  code  units  of  the  previous 
development,  while  columns  one  and  three  represent  traceability  between  design 
elements  and  code  units  of  the  final  modified  software.  The  implementation  of  design 
element  ‘A’  is  shown  as  traceable  to  code  unit  ‘M’  (and  vice-versa)  and  remains 
unchanged  by  the  modification.  The  implementation  of  design  element  ‘B’  has  been 
restructured  by  the  modification  and  is  now  traced  to  the  new  code  unit  ‘R’  in  addition  to 
the  previous  code  unit  ‘N’.  Design  element  ‘C’  was  implemented  in  code  unit  ‘O’  of  the 
previous  code,  but  has  now  been  removed  as  a  result  of  the  modification.  Traceability 
information  for  design  element  ‘D’  is  not  shown  for  either  the  previous  or  the  modified 
code.  The  intention  in  this  hypothetical  example  is  that  design  element  ‘D’  does  exist  as 
a  necessary  element  of  the  design  and  is  present  in  the  software  (as  verified  by 
requirements-based  testing),  but  its  traceability  information  was  simply  overlooked  in  the 
previous  development.  Also  for  the  purposes  of  this  example,  design  element  ‘D’  was 
not  associated  with  or  considered  during  the  software  modification  which  prevents 
anything  being  said  about  its  traceability  after  the  modification.  Code  unit  ‘Q’  is 
included  in  Table  3  to  represent  code  that  does  not  have  any  identified  backwards 
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traceability  to  design.  Again,  this  reappears  in  the  modified  software  because  it  was  not 
specifically  addressed  as  part  of  the  modification.  Design  element  ‘E’  is  a  new  feature 
that  is  introduced  as  part  of  the  modification  and  is  traced  to  code  unit  ‘S’. 


Design  Element 

Previous  Code  Unit 

Modified  Code  Unit 

A 

M 

M 

B 

N 

N 

R 

C 

0 

D 

? 

? 

? 

Q 

Q 

E 

s 

Tables.  Modified  Code  Traceability 


Consider  the  final  traceability  data  that  is  proposed  to  meet  certification 
requirements  in  the  two  parts  identified  in  Equation  17;  firstly  the  later  of  the  two  parts, 
that  which  was  generated  during  the  modification,  and  lastly  the  earlier  of  the  two  parts, 
that  which  was  generated  for  the  previous  development. 

Traceability  Data  =  (Traceability  Data^ )  U  (Traceability  Dataj^ ) 

Equation  17.  Software  Traceability  Evolution 

In  this  example,  the  software  modification  was  composed  of  three  facets: 
(i)  refactoring  of  design  element  ‘B’,  (ii)  removal  of  design  element  ‘C’  and  (hi)  addition 
of  design  element  ‘E’.  It  would  be  expected  that  the  traceability  data  for  design  element 
‘E’  will  be  properly  detailed  as  part  of  the  modification  effort  and  as  such,  satisfy  the  new 
certification  requirements.  We  propose  that  the  new  traceability  for  design  element  ‘B’ 
will  also  be  complete  after  the  modification,  although  this  will  be  somewhat  different 
from  the  traceability  of  design  element  ‘B’  for  the  previous  development.  Traceability 
for  design  element  ‘C’  will  not  exist,  nor  should  it,  for  the  modified  software. 
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The  traceability  data  from  the  previous  development  (Traceability  Datax 
in  Equation  17),  even  in  this  simple  scenario,  requires  a  morphism  that  is  a  composition 
of  identity,  partial  and  null  morphisms.  Consider  each  of  the  design  elements  and  code 
units  in  turn. 

The  traceability  data  for  design  element  ‘A’  for  the  previous  development 
can  probably  be  used  as-is  for  the  modified  software.  In  this  case  the  identity  morphism 
applies; 


^zi^Traceabilty^^  j  — >  Traceabilty^^ 

Equation  22.  Identity  Morphism  -  Code  Traceability 

The  software  developer  would  very  likely  use  the  previous  design  element 
‘B’  traceability  data  as  a  basis  for  establishing  code  traceability  in  the  modified  code.  If 
this  is  the  case,  the  final  traceability  of  design  element  ‘B’  will  be  a  composite  of  the  new 
traceability  data  to  code  unit  ‘R’  with  a  partial  morphism  of  the  previous  traceability  data. 
This  is  represented  in  Equation  23. 

Traceability  Data^  =  ^J^Traceabiltyg^  ^  Traceabiltyg^  j  U  (^Traceability  Datag_^  ^ 
Equation  23.  Partial  Morphism  -  Design  Element  ‘B’  Code  Traceability 

The  traceability  data  for  design  element  ‘C’  for  the  previous  development 
cannot  be  used  for  the  modified  software.  In  this  example,  the  design  element  and 
corresponding  code  unit  have  been  removed  from  the  modified  software.  Traceability 
data  for  this  portion  of  the  software  in  the  previous  development  is  no  longer  applicable 
to  the  final  modified  code.  Eurthermore,  as  it  is  not  part  of  the  modification  (by 
omission),  any  latent  reference  to  it  in  the  final  traceability  data  will  be  an  error.  In  this 
case  the  null  morphism  should  apply  as  in  Equation  24: 

^zi(Traceabilty^^  ^  ^  0 

Equation  24.  Null  Morphism  -  Code  Traceability 
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Design  element  ‘D’  and  code  unit  ‘Q’  do  not  have  any  identified 
traceability  data  from  previous  development  and  hence  nothing  can  be  said  with  certainty 
about  their  traceability  after  the  software  modification.  There  is  no  traceability 
information  for  them  and  therefore  nothing  to  subject  to  a  morphism  function. 

This  example  only  focuses  on  a  narrow  aspect  of  traceability  between 
design  and  code.  Traceability  between  other  software  products  is  also  needed,  ft  is  usual 
practice  to  have  all  of  the  traceability  information  collected  into  one  traceability  matrix  or 
database.  In  this  case,  the  reader  would  appreciate  that  identification  of  existing 
traceability  data  for  appropriate  reuse  becomes  an  increasingly  complex  problem  with 
variation  in  the  amount  of  partial  morphism  that  would  need  to  be  applied  to  different 
software  products. 

b.  Software  Testing 

The  design  elements  and  code  units  already  presented  in  Table  3  are  used 
again  for  the  following  analysis  of  test  results  for  the  modified  code.  We  again  treat  the 
latter  of  the  two  parts  of  the  complete  test  results  first,  before  discussing  the  reuse  of 
previous  development  test  results. 

Test  Logs  and  Results  =  (Test  Logs  and  Results^ )  U  (Test  Logs  and  Results ) 
Equation  19.  Software  Test  Results  Evolution 

We  again  assume  that  the  testing  of  modified  code  satisfies  the 
requirements  for  the  new  software  assurance  standard  (and  any  unmodified  code  that  may 
interface  the  modification).  This  assumption  equates  to  full  acceptance  of  code  unit  R 
and  S  test  logs  and  results,  and  partial  acceptance  or  possibly  full  acceptance  of  unit  M 
test  logs  and  results.  What  remains  is  to  determine  the  suitability  of  the  test  results  for 
the  previous  development  that  have  not  been  covered  as  part  of  the  modification.  N.B.: 
We  assume  here  that  the  certification  requirements  placed  on  testing  of  the  evolved 
software  has  changed  and  that  this  new  standard  is  more  stringent  in  its  requirements. 

The  degree  to  which  test  plans,  cases,  procedures,  input  data,  logs  and 
results  associated  with  code  unit  N  from  the  previous  development  can  be  reused  will 
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depend  on  the  manner  in  which  testing  was  conducted  for  code  units  N  and  R  after 
modification  of  the  software.  We  believe  it  to  be  likely  that  test  cases,  descriptions  and 
procedures  would  be  completely  redeveloped  for  the  modification  in  all  but  the  most 
trivial  modification  efforts.  This  belief  is  based  on  the  fact  that  design  element  B  has 
been  refactored  across  two  code  units  and  the  effort  required  to  plan  and  conduct 
completely  new  tests  would  be  justified  by  comparison  to  the  effort  required  to  develop 
only  some  new  tests  but  integrate  them  with  reused  existing  tests  and  results.  This 
assumption  is  highly  dependent  on  the  amount  of  refactoring,  but  we  believe  that 
refactoring  to  include  a  second  code  unit  would  be  significant  enough  to  justify  the 
assumption;  under  this  assumption  we  propose  that  the  morphism  to  be  applied  to 
existing  test  results  for  code  unit  N  would  be  a  null  morphism.  Testing  for  code  unit  O, 
like  traceability  for  it,  has  no  bearing  on  the  modified  software  and  should  also  be 
subjected  to  the  null  morphism. 

^zi(^Test  Logs  and  Results  ^  ^  0 
Equation  25  Null  Morphism  -Test  Logs  and  Results 

As  was  stated  in  the  previous  discussion  regarding  code  traceability,  the 
code  for  design  element  D  has  been  tested,  despite  not  having  traceability  data  to  identify 
which  code  unit  provides  the  implementation.  Test  results  could  be  obtained  without  the 
code  unit  identification  by  undertaking  requirements-based  integration  testing.  However, 
as  mentioned  in  §IILB,  there  are  other  testing  techniques  in  addition  to  requirements- 
based  testing  that  we  propose  would  be  the  requirements  of  the  newly  applied  software 
assurance  standard  S2.  In  this  case  the  mapping  of  test  logs  and  results  for  design 
element  D  will  be  the  partial  morphism. 

(5' (Test  Logs  and  Results^ )  ^  Test  Logs  and  ResultSi^j^ 

Equation  26.  Partial  Morphism  -  Design  Element  ‘D’  Test  Logs  and  Results 
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Once  again,  code  unit  ‘Q’  does  not  have  any  identified  design  element. 
Code  unit  Q  is  thus  unlikely  to  have  any  test  logs  and  results  from  the  previous 
development  and  so  again,  there  is  nothing  to  subjeet  to  a  morphism  function. 

The  preceding  discussion  pertaining  to  the  reuse  of  prior  development  test 
logs  and  results  only  covered  the  issue  of  existenee  of  the  software  products,  not  the 
suitability  of  the  produets  that  do  exist.  An  assessment  of  the  adequacy  of  the  previous 
test  logs  and  results  to  meet  the  new  standard  would  have  to  be  made.  The  measure  of 
adequaey  would  inelude  what  elass  (timing/performanee/loading  ete.)  of  testing  was 
conducted,  what  coverage  was  provided  (requirements/statement/state/range  -based),  and 
the  evaluation  eriteria.  It  is  likely  that  under  these  considerations,  the  degree  of  partial 
morphism  .for  many,  if  not  all,  previous  test  logs  and  results  would  be  further  reduced  by 
a  partial  morphism. 

c.  General  Considerations  for  Software  Product  Morphism 

The  preeeding  diseussion  has  not  distinguished  whether  the  assigned 
safety  level  of  the  modified  software  has  changed  from  the  assigned  safety  level  for  the 
previous  development.  In  the  cases  where  the  assigned  safety  level  has  been  increased, 
the  amount  of  software  product  from  the  previous  development  that  can  be  reused  for  the 
purposes  of  certification  of  the  modified  software  would  be  expected  to  be  significantly 
reduced.  It  would  be  probable  that  any  of  the  software  produets  to  be  reused  would  be 
subject  to  partial  morphism  and  require  additional  work  to  make  them  suitable  for  use  in 
the  new  certifieation  applieation. 
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IV.  RELATED  WORK 


A,  ARCHITECTURAL  TRANSFORMATION 

Grunske’s  work  on  the  semi-automatic  improvement  of  the  nonfunctional 
properties  of  software  using  hypergraph  transformations  [28]  is  founded  on  the  concept 
of  morphisms  to  modify  architectural  elements  during  the  design  stage  of  software 
development.  This  work  compliments  the  approach  proposed  in  this  thesis  by  providing 
a  technique  to  assess  a  given  system’s  architecture  for  its  ability  to  achieve  specified 
requirements  for  software  quality  such  as  system  safety.  The  technique  provides  an 
assessment  of  whether  the  software  architecture  complies  with  stated  requirements  for 
software  quality,  or  requires  alteration  to  meet  system  requirements. 

It  is  cost  effective  to  continually  conduct  evaluations  of  the  nonfunctional 
properties  of  a  system  from  the  earliest  possible  opportunity.  Grunske  proposes  a  method 
for  semi-automatically  conducting  a  comparison  of  nonfunctional  properties  of  a  system 
with  different  architectures  of  the  required  components.  The  comparison  begins  with  an 
architectural  specification  that  satisfies  the  entire  set  of  functional  requirements,  and  then 
continues  with  alternative  architectural  elements  that  are  available  in  the  analysis  tool.  If 
the  evaluation  determines  that  a  nonfunctional  requirement  is  unachievable  to  the  desired 
level,  such  as  safety,  then  the  architectural  specification  must  be  transformed  to  one  that 
achieves  the  quality  performance  requirements  without  compromising  the  attainment  of 
functional  requirements.  If  no  impediments  to  nonfunctional  properties  are  identified, 
then  the  development  process  can  proceed  using  the  originally  proposed  architectural 
specification. 

1,  Hierarchical  Typed  Hypergraphs 

The  work  is  based  on  Hierarchical  Typed  Hypergraph  (HTH)  theory.  Hypergraph 
theory  is  an  extension  of  graph  theory  that  permits  more  than  two  nodes  (vertices)  to  an 
edge.  Typed  hypergraphs  are  a  restriction  within  general  hypergraphs  that  stipulates 
node  and  hyperedge  types',  hyperedges  of  a  certain  type  may  only  connect  nodes  of 
certain  types.  Hierarchical  Typed  Hypergraphs  are  typed  hypergraphs  that  use 
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hyperedges  called  complex  hyperedges  that  are  themselves  hypergraphs.  This  permits  a 
recursive  construction  of  a  full  HTH  using  a  lesser  HTH,  hyperedges  and  nodes. 

A  HTH  is  characterized  by  the  tuple  (V,E,att,  lab,  ext,  cts)  where; 

F  is  a  finite  set  of  nodes  (vertices)  from  the  set  of  node  types  Ly, 

E  is  a  finite  set  of  hyperedges  from  the  set  of  hyperedge  types  Eg, 

att  is  the  attachment  function  to  assign  a  sequence  of  nodes  to  a  hyperedge, 

lab  is  the  labeling  function  for  nodes  and  hyperedges, 

ext  describes  a  sequence  of  external  nodes,  and 

cts  is  an  assignment  function  that  specifies  that  a  hyperedge  contains  a  HTH. 

2,  Unified  Modeling  Language  for  Real-Time 

Grunske  applies  his  theory  to  architectures  specified  using  the  Unified  Modeling 
Language  for  Real-Time  (UML-RT).  To  do  so,  he  derives  the  UML-RT  metaclass 
capsule  from  the  HTH  metaclass  hypergraph,  the  UML-RT  metaclass  port  (both  end  and 
relay  types)  to  the  HTH  metaclass  node,  and  the  UML-RT  metaclass  connector  to  the 
HTH  metaclass  hyperedge.  Figure  4  presents  these  relationships. 
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meta-levd  1  (UML  RT  stmcture) 


Figure  4.  Metaclass  Mapping  of  HTH  and  UML-RT  Elements  (From;  [28]) 

3,  Hierarchical  Typed  Hypergraphs  Transformations 

Hypergraph  transformation  is  achieved  by  identifying  sub-hypergraphs  within 
hypergraphs  to  enable  hypergraph  subtraction,  hypergraph  addition  to  construct  an 
expanded  hypergraph,  and  hypergraph  morphisms  of  attachments  between  nodes  and 
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hyperedges  to  complete  the  reconstruction.  A  HTH  morphism  (m)  from  one  HTH  (G)  to 
a  functionally  equivalent  HTH  (G’),  uses  pairs  of  mappings  for  nodes  and  hyperedges. 
For  instance,  m  :  G  ^  G '  =<  >  where  My  :V  ^V'  and  My.  :E  ^  E' . 

The  graphical  T-notation  represented  in  Figure  5  is  used  to  identify  the  operands 
of  the  transformation  as  follows; 

1 .  Ports  (nodes)  and  connections  (hyperedges)  in  the  lower  left  corner  of  the 
T  are  to  be  subtracted  from  the  original  HTH. 

2.  Ports  and  connections  (and  in  the  case  of  Figure  5,  new  additional 
components)  in  the  lower  right  corner  of  the  T  are  to  be  added  to  the  HTH. 

3.  Ports  above  the  T  are  retained  in  the  architectural  transformation. 

4.  Morphisms  of  the  attachments  between  ports  and  connections  reconfigure 
the  resultant  architecture  to  retain  functional  equivalence  with  the  original 
architecture. 

The  example  presented  in  Figure  5  shows  a  single  capsule  on  the  left  being 
replaced  with  a  subsystem  that  has  three  capsules^  on  the  right,  that  provide  their  output 
to  a  voting  capsule  that  chooses  which  of  the  messages  to  use  based  on  a  two-out-of-three 
voting  system. 


Figure  5.  Hypergraph  Graphical  T-Notation  (From;  [28]) 


7  Functionally  equivalent  to  the  original  subtracted  capsule  in  this  example,  but  that  need  not  be  the 
case. 
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4,  Evaluation  of  Quality  Characteristics 

Each  element  of  an  arehiteetural  speeifieation  is  annotated  with  the  relevant 
aspeets  of  the  quality  eharaeteristies  of  interest  and  the  eapsules  extended  with  analysis 
models  to  facilitate  the  arehiteetural  evaluation.  For  example,  to  evaluate  reliability  or 
safety,  the  elements  will  be  annotated  with  failure  data  (safe  and  unsafe),  and  extended 
with  a  fault  tree  analysis  model. 

Arehiteetural  transformations  ean  then  step  through  a  pre-defined  library  of 
possible  transformations  and  be  evaluated  for  performanee  against  the  nonfunetional 
properties  of  concern. 

B,  COMPUTER-AIDED  SOFTWARE  EVOLUTION 

Harn  [29]  extends  the  use  of  hypergraphs  to  a  relational  hypergraph  model  as  a 
means  for  formaly  defining  software  evolution.  This  was  done  by  relating  software 
evolution  objects  as  inputs  and  outputs  of  the  software  steps  that  use  and  produee  the 
objeets.  A  hypergraph  eapturing  this  relationship  is  defined  in  [29]  as  the  tuple 
(^N,E,I,0)  where; 

A  is  a  set  of  nodes  representing  software  evolution  objects, 

E  is  a  set  of  hyperedges  representing  steps  during  software  evolution, 

/  is  the  set  of  inputs  for  each  hyperedge,  and 

O  is  the  set  of  outputs  from  eaeh  hyperedge. 

In  a  relational  hypergraph,  every  input  node  on  a  hyperedge  is  either  a  primary,  or 
a  secondary  input  to  the  hyperedge,  and  the  output  of  the  step  is  dependent  on  all  input 
nodes.  Primary-input-driven  hypergraphs  relate  different  versions  of  the  same  software 
evolution  objeet,  while  seeondary-input-driven  hypergraphs  relate  different  software 
evolution  objects.  Figure  6  shows  an  example  of  a  relational  hypergraph  where  the 
output  software  produet  (P2.2)  is  produeed  as  a  new  (and  merged)  version  of  earlier 
software  produets  (Pl.l,  Pi2.2  and  P22.2)  and  is  also  dependent  on  another  software 
produet  (Rl.l). 
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A  primary-input-driven  path  addresses  the  evolution  history  of  a  software 
evolution  component  based  on  the  change  from  an  old  version  to  a  new 
version.  A  secondary-input-driven  path  addresses  the  evolution  rationale 
with  a  sequence  of  the  software  evolution  components.  Therefore,  these 
two  structures  form  the  relational  hypergraph  which  determines  not  only 
what  to  evolve  but  also  how  to  evolve  it.  [29] 


Figure  6.  Relational  Hypergraph  (From;  [29]) 


The  functional  notation  using  abstract  algebra  proposed  in  this  thesis  has  the 
following  similarities  to  the  relational  hypergraph  model  presented  in  Ham’s  dissertation; 

1 .  The  use  of  the  function  operator  for  software  development  activities 
presented  in  Equation  5  is  similar  to  the  use  of  hyperedges  as  steps  of 
software  evolution. 

2.  Software  products  being  identified  as  the  operands  of  development 
activities  is  equivalent  to  nodes  representing  input  software  evolution 
objects  to  software  evolution  steps. 

3.  Software  product  being  produced  as  the  outputs  from  development 
activities  equate  nodes  representing  software  evolution  objects  as  outputs 
from  steps. 

Ham’s  work  contrasts  the  work  of  this  thesis,  which  is  a  technique  for  the  analysis 
of  the  suitability  of  previously  developed  software  product  for  reuse,  when  the  software 
and  the  standards  being  applied  are  both  evolving.  As  previously  mentioned,  this  reuse 
may  be  intentional  or  be  regarded  as  salvage  when  the  reuse  of  the  software  product  is 
not  preplanned. 
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The  work  in  [29]  covers  the  deliberate  reuse  of  software  evolution  objects  during 
software  development.  Although  Harn  does  not  mention  software  assurance  certification 
as  one  of  the  steps  of  software  evolution,  the  tool  that  was  developed  could  be  extended 
to  incorporate  certification. 

C.  F/RF-lllC  AGM-142E/1760  INTEGRATION 

The  AGM-142E/1760  Integration  as  part  of  the  F/RF-lllC  Block  C4  SCS 
upgrade  was  initially  thought  to  be  candidate  for  application  of  the  technique  proposed  in 
this  thesis.  Unfortunately,  the  software  for  the  System  Interface  Processor  (SIP)  is  not  an 
example  of  legacy  safety-critical  software  that  was  having  its  certification  basis  upgraded 
during  modification. 

AGM-142E/1760  Integration  involves  both  the  modification  of  software  for 
several  processors^  already  embedded  in  the  E/RE-1 1 1C  and  incorporation  of  a  SIP.  The 
SIP  provides  the  necessary  interface  between  the  MC  and  AGM-I42E  loaded  onto 
aircraft  Weapon  Stations  (WS)  which  was  not  possible  through  the  existing  SMP 
interface  to  the  WS.  The  certification  basis  for  the  existing  subsystems  was  to  remain  a 
tailored  version  of  MIE-STD-498,  while  DO-178B  (software  level  B)  was  proposed  as 
the  certification  basis  for  the  SIP  software.  Figure  7  is  a  simplified  block  diagram 
showing  the  data  and  control  connections  between  system  components  post  integration. 

The  software  units  within  the  SIP  are  the  operating  system,  bus  driver,  discrete 
input/output  driver,  analog  to  digital  converter  driver  which  were  COTS  products  and  the 
OFP  which  was  a  completely  new  software  development. 


8  Mission  Computer  (MC),  Stores  Management  Proeessor  (SMP)  Bus  Sub-System  Integration  Unit, 
and  System  Funetion  Proeessor. 
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Figure  7.  AGM-142E  Integration  -  Simplified  Bloek  Diagram 


Certifying  COTS  eomponents  as  part  of  a  DO-178B  eertified  system  is  a  related 
but  distinet  problem  from  that  of  upgrading  the  eertifieation  basis  for  previously 
developed  software.  Certifieation  of  COTS  produets  is  well  reeognized  within  DO-178B 
and  guidanee  to  aehieve  the  desired  eertifieation  is  ineluded  within  the  standard. 

The  possibility  of  applying  abstraet  algebra  to  deseribe  the  development  of  the 
OFF  existed  but  was  not  explored.  The  reason  for  not  investigating  this  projeet  further 
was  that  the  OFF  was  a  new  development  and  henee  no  software  produets  existed  from  a 
previous  development  to  whieh  the  proposed  morphisms  eould  be  applied. 
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V. 


CONCLUSION 


A,  KEY  FINDINGS  AND  ACCOMPLISHMENTS 

In  this  thesis  we  investigated  the  ADF  practice  of  certifying  modified  legacy 
software  to  DO-178B  where  the  modification  is  developed  in  compliance  with  DO-178B, 
but  where  the  software  was  previously  developed  to  some  other  standard.  The  ADF 
practice  is  based  on  an  extension  of  the  FAA  policy  for  certification  of  software  to  DO- 
178B,  which  has  been  previously  developed  to  an  earlier  version  of  the  DO- 178  standard, 
and  subsequently  modified  in  such  a  manner  as  to  be  compliant  with  DO-178B. 

The  original  scope  of  the  research  was  to  examine  the  objectives  of  DO-178B  and 
make  a  comparison  with  another  software  assurance  or  development  standard  to  see  how 
much  of  the  previous  software  development  activities  and  products  could  possibly 
comply  with  the  certification  requirements  of  DO-178B.  If  sufficient  software  products 
and  activities  are  found  to  be  satisfactory,  then  some  form  of  automation  of  any  technique 
to  reuse  the  software  products  from  a  previous  development  would  be  necessary  to  make 
such  reuse  practical.  A  necessary  first  step  in  achieving  automation  of  any  software 
product  reuse  is  to  define  a  mathematical  description  of  software  evolution  to  enable  the 
mapping  of  applicable  software  activities  and  products  between  software  standards. 
Thus,  deriving  such  a  mathematical  description  of  software  development  and  evolution 
became  the  primary  focus  of  this  work. 

In  this  thesis,  we  introduced  an  algebraic  model  for  software  development  as  a 
means  to  apply  a  systematic  approach  to  define  software  development  and  evolution. 
Our  algebraic  model  introduces  the  application  of  abstract  algebra  to  software 
development  by  defining  software  products  as  the  set  of  symbols,  and  software 
development  activities  as  the  set  of  operations. 

The  abstract  algebra  model  has  a  function  notation  that  defines  software 
development  activities.  This  function  notation  was  developed  for  a  single  activity, 
software  development,  and  software  evolution. 
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Equations.  Single  Activity 

m 

Aj 

^■=1 

Equation  7.  Software  Development 

X-  =  (]a,(B,)(iua,(B,) 

s=\  v“'  y 

Equation  8.  Software  Evolution 

To  assist  in  the  efficient  achievement  of  recertification  against  new  or  evolving 
standards,  we  demonstrated  the  application  of  morphism  to  map  software  products  for 
reuse  during  software  evolution.  Three  general  functions  that  were  presented  are  (i)  the 
identity  function  for  software  product  that  can  be  reused  without  additional  effort,  (ii)  the 
partial  function  for  software  product  that  is  only  partially  reusable  and  (iii)  the  null 
function  for  software  product  that  cannot  be  reused  to  achieve  recertification. 

<l>[cc,)^cc, 

Equation  9.  Identity  Morphism 
^  da^ 

Equation  10.  Partial  Morphism 
«»(«,)  ^0 

Equation  1 1 .  Null  Morphism 

Whilst  the  model  presented  in  this  thesis  is  yet  to  be  completed,  and  specific 
morphisms  for  mapping  software  products  are  still  to  be  identified,  the  approach  has  been 
well  received  and  shows  promise  of  satisfying  the  basis  for  automation  of  recertification. 
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B,  CONCLUDING  REMARKS 

A  comment  raised  towards  the  end  of  this  work  was  in  essence  a  question  of 
“does  it  really  matter  once  a  system  is  in  service?”  Mapping  prior  activities  and  software 
products  to  a  new  software  assurance  standard  does  not,  on  its  own,  make  a  safety-critical 
system  any  safer;  to  make  software-intensive  systems  safer,  the  code  must  be  improved. 
This  is  a  valid  assertion  but  the  reply  to  this  point  is  that  software  safety  assurance  is  an 
attempt  to  obtain  a  measure  of  the  confidence  in  the  safety  aspects  of  a  software-intensive 
system. 

Establishing  the  current  level  of  software  safety  assurance  based  on  existing 
available  evidence  permits  program  managers  to  make  justifiable  decisions  concerning 
the  continued  operation,  or  otherwise,  of  aircraft  with  or  without  software  upgrades.  The 
first  step  in  the  decision-making  process  is  to  determine  the  eurrent  level  of  software 
safety  assurance.  If  the  finding  gives  the  certification  authority  sufficient  grounds  for 
claiming  an  acceptable  level  of  system  safety  has  been  achieved,  then  the  system  can 
continue  to  be  operated  without  any  further  expenditure  or  effort.  This  prevents 
unwarranted  expenditure  of  resources  on  software  evolution  that  is  not  necessary  from 
the  software  safety  certification  perspective.  If  the  finding  does  not  satisfy  the 
certification  authority,  then  the  following  options  can  be  explored  to  reach  a  decision. 
The  first  option  is  always  do  nothing  and  continue  to  operate  the  system  as-is.  This 
sounds  unpalatable  when  talking  about  safety,  but  is  none-the-less  one  of  the  options.  A 
decision  to  take  this  option  can  either  be  an  educated  or  uneducated  one;  determining  the 
current  level  of  software  safety  assurance  gives  the  deeision  maker  the  opportunity  to 
make  an  educated  decision.  The  second  option  is  to  modify  the  software  within  the 
system  to  raise  the  software  safety  assurance  level.  The  third  option  is  to  discontinue 
operation  of  the  existing  system  and  replace  it.  These  last  two  choices  are  also  better 
made  with  a  clear  understanding  of  a  system’s  existing  level  of  software  assurance.  We 
hope  that  the  technique  proposed  in  this  thesis  will  be  effective  for  obtaining  evidence 
from  the  existing  software  products,  upon  which  an  educated  choice  about  continued 
system  operation  or  evolution  can  be  made. 
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C.  FUTURE  WORK 

1.  Determine  the  Morphisms  Required  for  Activities  and  Products 

The  immediate  work  to  be  eontinued  is  the  identification  of  a  complete  set  of 
morphisms  needed  to  map  the  requirements  of  a  legacy  standard  such  as  MIL-STD-498 
to  the  objectives  of  the  desired  standard  DO-178B.  Completeness  of  the  set  of 
morphisms  will  be  achieved  when  the  requirements  of  the  desired  standard  that  can  be 
achieved  with  the  legacy  software  products  are  identified  by  appropriate  morphisms. 

2,  Case  Studies 

A  candidate  for  investigation  and  application  of  the  proposed  technique  is  the 
AF/A-18  software  development  being  undertaken  by  the  RAAF  at  the  Tactical  Fighter 
Weapon  System  Support  Unit.  The  MC  and  SMP  OFF  for  the  AF/A-18  is  an  active  area 
of  the  evolution  of  legacy  software  which  was  originally  developed  to  MIL-STD-498. 
The  aircraft  will  be  subject  to  continued  software  maintenance  and  modernization  and  has 
sufficient  remaining  useful  life  to  warrant  the  investigation  into  upgrading  the  software 
certification  basis. 

Other  possibilities  within  the  ADF  might  be  the  RAAF  F/RF-1 1 1C  MC  and  SMP 
software  development  which  currently  have  a  MIL-STD-498  certification  basis;  or 
software  support  for  the  Royal  Australian  Navy’s  Super  Seasprite  helicopters,  FFG 
Frigates  or  Collins  class  submarines. 

3.  Automation 

We  feel  that  the  model  and  notation  for  the  representation  of  software 
development,  evolution  and  recognition  of  prior  software  product  presented  in  this  thesis 
is  amenable  to  automation  through  the  development  of  extensions  to  existing  software 
development  tools. 

4,  Different  Combinations  of  Other  Standards 

The  original  focus  of  this  thesis  was  a  comparative  analysis  of  the  DO-178B 
objectives  and  MIL-STD-498  requirements  to  determine  the  amount  of  MIL-STD-498 
compliant  software  product  that  would  likely  be  acceptable  for  certification  of  software  to 
DO-178B. 
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5,  Application  to  Other  Domains  and  Dimensions 

The  methodology  presented  in  this  thesis  can  be  applied  to  other  domains  that 
have  safety-critical  concerns.  These  include  rail  system  automation,  nuclear  power  plant 
control  and  medical  devices. 

Another  dimension  of  high  assurance  software  that  might  benefit  from  the 
application  of  this  technique  is  software  security  certification. 

It  is  possible  that  the  approach  introduced  here  may  also  be  applied  to  other 
engineering  disciplines  that  are  concerned  with  recertification. 

6,  Cost-Benefit  Analysis  With  Respect  to  Software/System  Age 

Achieving  certification  to  a  new  standard  is  a  considerable  effort  with  significant 

costs.  It  is  important  therefore,  to  obtain  an  acceptable  ROT  One  could  explore  whether 
the  cost  of  recertification  would  diminish  as  a  system  gets  closer  to  the  end  of  its  service 
life.  Cost-benefit  analysis  of  recertification  of  a  system  as  a  function  of  the  remaining 
useful  life  of  the  system  is  another  possibility  for  future  work. 
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APPENDIX:  EXTRACTS  FROM  SELECTED  STANDARDS 


A,  EXTRACTS  FROM  RTCA  DO-178B 

12.0  ADDITIONAL  CONSIDERATIONS 

12.1  Use  of  Previously  Developed  Software 

The  guidelines  of  this  subsection  discuss  the  issues  associated  with  the  use 
of  previously  developed  software,  including  assessment  of  modifications, 
the  effect  of  changing  an  aircraft  installation,  application  environment,  or 
development  environment,  upgrading  a  development  baseline,  and  SCM 
and  SQA  considerations.  The  intention  to  use  previously  developed 
software  is  stated  in  the  Plan  for  Software  Aspects  of  Certification. 

12.1.1  Modifications  to  Previously  Developed  Software 

This  guidance  discusses  modifications  to  previously  developed  software 
where  the  outputs  of  the  previous  software  life  cycle  processes  comply 
with  this  document.  Modification  may  result  from  requirement  changes, 
the  detection  of  errors,  and/or  software  enhancements.  Analysis  activities 
for  proposed  modifications  include; 

a.  The  revised  outputs  of  the  system  safety  assessment  process  should  be 
reviewed  considering  the  proposed  modifications. 

b.  If  the  software  level  is  revised,  the  guidelines  of  paragraph  12. 1 .4  should 
be  considered. 

c.  But  the  impact  of  the  software  requirements  changes  and  the  impact  of 
software  architecture  changes  should  be  analyzed,  including  the 
consequences  of  software  requirement  changes  upon  other  requirements 
and  coupling  between  several  software  components  that  may  result  in 
reverification  effort  involving  more  than  the  modified  area. 

d.  The  area  affected  by  change  should  be  determined.  This  may  be  done  by 
data  flow  analysis,  control  flow  analysis,  timing  analysis  and  traceability 
analysis. 

e.  Areas  affected  by  the  change  should  be  reverified  considering  the 
guidelines  of  section  6. 

12.1,4  Upgrading  a  Development  Baseline 

Guidelines  follow  for  software  whose  software  life  cycle  data  from  a 
previous  application  are  determined  to  be  inadequate  or  do  not  satisfy  the 
objectives  of  this  document,  due  to  the  safety  objectives  associated  with  a 
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new  application.  These  guidelines  are  intended  to  aid  in  the  acceptance 
of; 


•  Commercial  off-the-shelf  software. 

•  Airborne  software  developed  to  other  guidelines. 

•  Airborne  software  developed  prior  to  the  existence  of  this 
document. 

•  Software  previously  developed  to  this  document  at  a  lower 
software  level 

Guidance  for  upgrading  a  development  baseline  includes: 

a.  The  objectives  of  this  document  should  be  satisfied  while  taking  advantage 
of  software  lifecycle  data  of  the  previous  development  that  satisfy  the 
objectives  for  the  new  application. 

b.  Software  aspects  of  certification  should  be  based  on  the  failure  conditions 
and  software  level(s)  as  determined  by  the  system  safety  assessment 
process.  Comparison  to  failure  conditions  of  the  previous  application  will 
determine  areas  which  may  need  to  be  upgraded. 

c.  Software  life  cycle  data  from  a  previous  development  should  be  evaluated 
to  ensure  that  the  software  verification  process  objectives  of  the  software 
level  are  satisfied  for  the  new  application. 

d.  Reverse  engineering  may  be  used  to  regenerate  software  life  cycle  data 
that  is  inadequate  or  missing  in  satisfying  the  objectives  of  this  document. 
In  addition  to  producing  the  software  product,  additional  activities  may 
need  to  be  performed  to  satisfy  the  software  verification  process 
objectives. 

e.  If  use  of  product  service  history  is  planned  to  satisfy  the  objectives  of  this 
document  in  upgrading  a  development  baseline,  the  guidelines  of 
paragraph  12.3.5  should  be  considered. 

f.  The  applicant  should  specify  the  strategy  for  accomplishing  compliance 
with  this  document  in  the  Plan  for  Software  Aspects  of  Certification. 

12,1.5  Software  Configuration  Management  Considerations 

If  previously  developed  software  is  used,  the  software  configuration 

management  process  for  the  new  application  should  include,  in  addition  to 

the  guidelines  of  section  7: 

a.  Traceability  from  the  software  product  and  software  life  cycle  data  of  the 
previous  application  to  the  new  application. 

b.  Change  control  that  enables  problem  reporting,  problem  resolution,  and 
tracking  of  changes  to  software  components  used  in  more  than  one 
application. 
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12.1.6  Software  Quality  Assurance  Considerations 

If  previously  developed  software  is  used,  the  software  quality  assurance 
process  for  the  new  application  should  include,  in  addition  to  the 
guidelines  of  section  8; 

a.  Assurance  that  the  software  components  satisfy  or  exceed  the  software  life 
cycle  criteria  of  the  software  level  for  the  new  application. 

b.  Assurance  that  changes  to  the  software  life  cycle  processes  are  stated  in 
the  software  plans. 

12.3.5  Product  Service  History 

If  equivalent  safety  for  the  software  can  be  demonstrated  by  the  use  of  the 
software’s  product  service  history,  some  certification  credit  may  be 
granted.  The  acceptability  of  this  method  is  dependent  on; 

•  Configuration  management  of  the  software. 

•  Effectiveness  of  problems  reporting  activity. 

•  Stability  and  maturity  of  the  software. 

•  Relevance  of  product  service  history  environment. 

•  Actual  error  rates  and  product  service  history. 

•  Impact  of  modifications. 

Guidance  for  the  use  of  product  service  history  includes; 

a.  The  applicant  should  show  that  the  software  and  associated  evidence  used 
to  comply  with  system  safety  objectives  have  been  under  configuration 
management  throughout  the  product  service  history. 

b.  The  applicant  should  show  that  the  problem  reporting  during  the  product 
service  history  provides  assurance  that  representative  data  is  available  and 
that  in-service  problems  were  reported  and  recorded,  and  are  retrievable. 

c.  Configuration  changes  during  the  product  service  history  should  be 
identified  and  the  effect  analyzed  to  confirm  the  stability  and  maturity  of 
the  software.  Uncontrolled  changes  to  the  Executable  Object  Code  during 
the  product  service  history  may  invalidate  the  use  of  product  service 
history. 

d.  The  intended  software  usage  should  be  analyzed  to  show  the  relevance  of 
the  product  service  history. 

e.  If  the  operating  environments  of  the  existing  and  proposed  applications 
differ,  additional  software  verification  should  confirm  compliance  with  the 
system  safety  objectives. 
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f.  The  analysis  of  configuration  changes  and  product  service  history 
environment  may  require  the  use  of  software  requirements  and  designed 
data  to  confirm  the  applicability  of  the  product  service  history 
environment. 

g.  If  the  software  is  a  subset  of  the  software  that  was  active  during  the  service 
period,  Then  analysis  should  confirm  the  equivalency  of  the  new 
environment  with  the  previous  environment,  and  determine  those  software 
components  that  were  not  executed  during  normal  operation 

Note;  Additional  verification  may  be  needed  to  confirm  compliance  with 
the  system  safety  objectives  for  those  components. 

h.  The  problem  report  history  should  be  analyzed  to  determine  how  safety- 
related  problems  occurred  and  which  problems  were  corrected. 

i.  Those  problems  that  are  indicative  of  an  inadequate  process,  such  as 
design  or  code  errors,  should  be  indicated  separately  from  those  whose 
cause  are  outside  the  scope  of  this  document,  such  as  hardware  or  system 
requirements  errors. 

j .  The  data  described  above  and  these  items  should  be  specified  in  the  Plan 
for  Software  Aspects  of  Certification; 

(1)  Analysis  of  the  relevance  of  the  product  service  history 
environment. 

(2)  Length  of  service  period  and  rationale  for  calculating  the 
number  of  hours  in  the  service,  including  factors  such  as 
operational  modes,  the  number  of  independently  operating 
copies  in  the  installation  and  in  service,  and  the  definition  of 
“  normal  operation”  and  “  normal  operation  time”. 

(3)  Definition  of  what  was  counted  as  an  error  and  rationale  for 
that  definition. 

(4)  Proposed  acceptable  error  rates  and  rationale  for  the  product 
service  history  period  in  relation  to  the  system  safety  and 
proposed  error  rates. 

k.  If  the  error  rate  is  greater  than  that  identified  in  the  plan,  these  errors 
should  be  analyzed  and  the  analyses  reviewed  with  the  certification 
authority. 
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ANNEX  A  (of  DO-178B) 

PROCESS  OBJECTIVES  AND  OUTPUTS  BY  SOFTWARE  LEVEL 


Objective 

Applic 

- 

Output 

CC  by 

ability  by 

S/W  Level 

S/W  Level 

Description 

Ref. 

A 

B 

c 

D 

Description 

Ref 

A 

B 

c 

Software  Planning  Process 

1 

Software  development  and 

4.1a 

PSAC 

11.1 

1 

1 

1 

1 

integral  processes  activities 

4.3 

SDP 

11.2 

1 

1 

2 

2 

are  defined. 

O 

o 

o 

o 

SVP 

11.3 

1 

1 

2 

2 

SCMP 

11.4 

1 

1 

2 

2 

SQAP 

11.5 

1 

1 

2 

2 

2 

Transition  criteria,  inter¬ 

4.1b 

relationships  and  sequencing 

4.3 

O 

o 

o 

among  processes  are 
defined. 

3 

Software  life  cycle 
environment  is  defined. 

4.1c 

o 

o 

o 

4 

Additional  considerations 
are  addressed. 

4.1d 

o 

o 

o 

o 

5 

Software  development 

4.1e 

SAV  Requirements  Stds 

11.2 

1 

1 

2 

standards  are  defined. 

o 

o 

o 

SAV  Design  Stds 

11.3 

1 

1 

2 

SAV  Code  Stds 

11.4 

1 

1 

2 

6 

Software  plans  comply  with 

4.1f 

o 

o 

o 

SQA  Records 

11.19 

2 

2 

2 

RTCADO-178B. 

4.6 

SAV  Verification  Results 

11.14 

2 

2 

2 

7 

Software  plans  are 

4.1g 

o 

o 

o 

SQA  Records 

11.19 

2 

2 

2 

coordinated. 

4.6 

SAV  V erification  Results 

11.14 

2 

2 

2 

Software  Development  Processes 

8 

High-level  requirements  are 
developed. 

5.1.1a 

o 

o 

o 

o 

SAV  Requirements  Data 

11.9 

1 

1 

1 

1 

9 

Derived  high-level 
requirements  are  defined. 

5.1.1b 

o 

o 

o 

o 

SAV  Requirements  Data 

11.9 

1 

1 

1 

1 

10 

Software  architecture  is 
developed. 

5.2.1a 

o 

o 

o 

o 

Design  Description 

11.10 

1 

1 

2 

2 

11 

Low-level  requirements  are 
developed. 

5.2.1a 

o 

o 

o 

o 

Design  Description 

11.10 

1 

1 

2 

2 

12 

Derived  low-level 
requirements  are  developed. 

5.2.1b 

o 

o 

o 

o 

Design  Description 

11.10 

1 

1 

2 

2 
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Objective 

Applic¬ 
ability  by 
S/W  Level 

Output 

CC  by 
S/W  Levei 

Description 

Ref. 

A 

B 

c 

D 

Description 

Ref 

A 

B 

c 

D 

13 

Source  Code  is  developed. 

5.3.1a 

O 

o 

o 

o 

Source  Code 

11.11 

1 

1 

1 

1 

14 

Executable  Object  Code  is 
produced  and  integrated  in 
the  target  computer. 

5.4.1a 

O 

o 

o 

o 

Executable  Object  Code 

11.12 

1 

1 

1 

1 

Verification  of  Outputs  of  Software  Requirements  Process 

15 

Software  high-level 
requirements  comply  with 
system  requirements. 

6.3.1a 

• 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

2 

16 

High-level  requirements  are 
accurate  and  consistent. 

6.3.1b 

• 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

2 

17 

High-level  requirements  are 
compatible  with  target 
computer. 

6.3.1c 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

18 

High-level  requirements  of 
verifiable. 

6.3. Id 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

19 

High-level  requirements 
conform  to  standards. 

6.3. le 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

20 

High-level  requirements  are 
traceable  to  system 
requirements. 

6.3. If 

o 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

2 

21 

Algorithms  are  accurate. 

6.3. Ig 

• 

• 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

Verification  of  Outputs  of  Software  Design  Process 

22 

Low-level  requirements 
comply  with  high-level 
requirements. 

6.3.2a 

• 

• 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

23 

Lower-level  requirements 
are  accurate  and  consistent. 

6.3.2b 

• 

• 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

24 

Low-level  requirements  are 
compatible  with  target 
computer 

6.3.2c 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

25 

Level-level  requirements  are 
verifiable. 

6.3.2d 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

26 

Level-level  requirements 
conform  to  standards. 

6.3.2e 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 
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Objective 

Applic¬ 
ability  by 
S/W  Level 

Output 

CC  by 
S/W  Level 

Description 

Ref. 

A 

B 

c 

D 

Description 

Ref 

A 

B 

c 

D 

27 

Level-level  requirements  are 
traceable  to  high-level 
requirements. 

6.3.2f 

O 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

28 

Algorithms  are  accurate. 

6.3.2g 

• 

• 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

29 

Software  architecture  is 
compatible  with  high-level 
requirements. 

6.3.3a 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

30 

Software  architecture  is 
consistent. 

6.3.3b 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

31 

Software  architecture  is 
compatible  with  target 
computer. 

6.3.3c 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

32 

Software  architecture  is 
verifiable. 

6.3.3d 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

33 

Software  architecture 
conforms  to  standards. 

6.3.3e 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

34 

Software  partitioning 
integrity  is  confirmed. 

6.3.3f 

• 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

2 

Verification  of  Outputs  of  Software  Coding  &  Integration  Processes 

35 

Source  Code  complies  with 
low-level  requirements. 

6.3.4a 

• 

• 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

36 

Source  Code  complies  with 
software  architecture. 

6.3.4b 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

37 

Source  Code  is  verifiable. 

6.3.4c 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

38 

Source  Code  conforms  to 
standards. 

6.3.4d 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

39 

Source  Code  is  traceable  to 
low-level  requirements. 

6.3.4e 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

40 

Source  Code  is  accurate  and 
consistent. 

6.3.4f 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

41 

Output  of  integration 
process  is  complete  and 
correct. 

6.3.5 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 
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Objective 

Applic 

- 

Output 

CC  by 

ability  by 

S/W  Level 

S/W  Level 

Description 

Ref. 

A 

B 

c 

D 

Description 

Ref 

A 

B 

c 

Testing  of  Outputs  of  Integration  Process 

42 

Executable  Object  Code 

6.4.2.1 

O 

O 

o 

o 

SAV  Verification  Cases  and 

11.13 

1 

1 

2 

2 

complies  with  high-level 

6.4.3 

Procedures 

requirements. 

SAV  Verification  Results 

11.14 

2 

2 

2 

2 

43 

Executable  Object  Code  is 

6.4.2.2 

O 

o 

o 

o 

SAV  Verification  Cases  and 

11.13 

1 

1 

2 

2 

robust  with  high-level 

6.4.3 

Procedures 

requirements. 

SAV  V erification  Results 

11.14 

2 

2 

2 

2 

44 

Executable  Object  Code 

6.4.2.1 

• 

• 

o 

SAV  Verification  Cases  and 

11.13 

1 

1 

2 

complies  with  low-level 

6.4.3 

Procedures 

requirements. 

SAV  V erification  Results 

11.14 

2 

2 

2 

45 

Executable  Object  Code  is 

6.4.2.2 

• 

o 

o 

SAV  Verification  Cases  and 

11.13 

1 

1 

2 

robust  with  low-level 

6.4.3 

Procedures 

requirements. 

SAV  Verification  Results 

11.14 

2 

2 

2 

46 

Executable  Object  Code  is 

6.4.3a 

o 

o 

o 

o 

SAV  Verification  Cases  and 

11.13 

1 

1 

2 

2 

compatible  with  target 

Procedures 

computer. 

SAV  Verification  Results 

11.14 

2 

2 

2 

2 

Verification  of  Verification  Process  Results 

47 

Test  procedures  are  correct. 

6.3.6b 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

48 

Test  results  are  correct  and 
discrepancies  explained. 

6.3.6c 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

49 

Test  coverage  of  high-level 
requirements  is  achieved. 

6.4.4.1 

• 

o 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

2 

50 

Test  coverage  of  low-level 
requirements  is  achieved. 

6.4.4.1 

• 

o 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

51 

Test  coverage  of  software 
structure  (modified 
condition/decision)  is 
achieved. 

6.4.4.2 

• 

SAV  Verification  Results 

11.14 

2 

52 

Test  coverage  of  software 

6.4.4.2a 

• 

• 

SAV  Verification  Results 

11.14 

2 

2 

structure  (decision 
coverage)  is  achieved. 

6.4.4.2b 

53 

Test  coverage  of  software 

6.4.4.2a 

• 

• 

o 

SAV  Verification  Results 

11.14 

2 

2 

2 

structure  (statement 
coverage)  is  achieved. 

6.4.4.2b 
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Objective 

Applic 

- 

Output 

CC  by 

ability  by 

S/W  Level 

S/W  Level 

Description 

Ref. 

A 

B 

c 

D 

Description 

Ref 

A 

B 

c 

D 

54 

Test  eoverage  of  software 
stmeture  (data  eoupling  and 
eontrol  eoupling)  is 
aehieved. 

6.4.4.2e 

• 

• 

o 

SAV  Verifieation  Results 

11.14 

2 

2 

2 

Software  Configuration  Management  Proeess 

55 

Configuration  items  are 

7.2.1 

O 

o 

o 

o 

SAV  Configuration 

11.18 

2 

2 

2 

2 

identified. 

Management  Reeords 

56 

Baselines  and  traee  ability 

7.2.2 

O 

o 

o 

o 

SAV  Configuration  Index 

11.16 

1 

1 

1 

1 

are  established. 

SAV  Configuration 
Management  Reeords 

11.18 

2 

2 

2 

2 

57 

Problem  reporting,  ehange 

7.2.3 

o 

o 

o 

o 

Problem  Reports 

11.17 

2 

2 

2 

2 

eontrol,  ehange  review,  and 

7.2.4 

SAV  Configuration 

11.18 

2 

2 

2 

2 

eonfiguration  status 

7.2.5 

Management  Reeords 

aeeounting  and  established. 

7.2.6 

58 

Arehive,  retrieval,  and 

7.2.7 

o 

o 

o 

o 

SAV  Configuration 

11.18 

2 

2 

2 

2 

release  are  established. 

Management  Reeords 

59 

Software  load  eontrol  is 

7.2.8 

o 

o 

o 

o 

SAV  Configuration 

11.18 

2 

2 

2 

2 

established. 

Management  Reeords 

60 

Software  life  eyele 

7.2.9 

o 

o 

o 

o 

SAV  Life  Cyele 

11.15 

1 

1 

1 

2 

environment  eontrol  is 

Environment  Configuration 

established. 

Index 

SAV  Configuration 
Management  Reeords 

11.18 

2 

2 

2 

2 

Software  Quality  Assuranee  Proeess 

61 

Assuranee  is  obtained  that 

8.1a 

• 

0 

0 

0 

SAV  Quality  Assuranee 

11.19 

2 

2 

2 

2 

software  development  and 
integral  proeesses  eomply 
with  approved  software 
plans  and  standards. 

Reeords 

62 

Assuranee  is  obtained  that 

8.1b 

0 

SAV  Quality  Assuranee 

11.19 

2 

2 

transition  eriteria  for  the 
software  lifeeyele  proeesses 
are  satisfied. 

Reeords 

63 

Software  eonformity  review 

8.1e 

• 

• 

• 

• 

SAV  Quality  Assuranee 

11.19 

2 

2 

2 

2 

is  eondueted. 

8.3 

Reeords 
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Objective 

Applic¬ 
ability  by 
S/W  Level 

Output 

CC  by 
S/W  Level 

Description 

Ref 

A 

B 

c 

D 

Description 

Ref 

A 

B 

C 

D 

Certification  Liaison  Process 

64 

Communication  and 
understanding  between  the 
applicant  and  the 
certification  authority  is 
established. 

9.0 

O 

o 

o 

o 

PSAC 

11.1 

1 

1 

1 

1 

65 

The  means  of  compliance  is 
proposed  and  agreement 
with  the  Plan  for  Software 
Aspects  of  Certification  is 
obtained. 

9.1 

O 

o 

o 

o 

PSAC 

11.1 

1 

1 

1 

1 

66 

Compliance  substantiation  is 
provided 

9.2 

o 

o 

o 

o 

Software  Accomplishment 
Summary 

Software  Configuration 

Index 

11.20 

11.16 

1 

1 

1 

1 

Legend 

o 

The  objective  should  be  satisfied. 

• 

The  objective  should  be  satisfied  with  independence. 

Blank 

Satisfaction  of  objective  is  at  applicant’s  discretion. 

1 

Data  satisfies  the  objectives  of  Control  Category  1  (CCl). 

2 

Data  satisfies  the  objectives  of  Control  Category  2  (CC2). 

Table  4.  Objectives,  Activities,  Outputs  and  Data  Control  Categories 
(After;  15,  Tables  A-1  through  A- 10) 


B,  EXTRACTS  FROM  MIL-STD-498 

4.2.2  Standards  for  software  products.  The  developer  shall  develop  and 
apply  standards  for  representing  requirements,  design,  code,  test  cases,  test 
procedures,  and  test  results.  These  standards  shall  be  described  in,  or 
referenced  from,  the  software  development  plan. 

5.16  Software  quality  assurance.  The  developer  shall  perform  software 
quality  assurance  in  accordance  with  the  following  requirements. 

Note;  If  a  system  or  CSCI  is  developed  in  multiple  builds,  the  activities 
and  software  products  of  each  build  should  be  evaluated  in  the  context  of 
the  objectives  established  for  that  build.  An  activity  or  software  product 
that  meets  those  objectives  can  be  considered  satisfactory  even  though  it  is 
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missing  aspects  designated  for  later  builds.  Planning  for  software  quality 
assurance  is  included  in  software  development  planning  (see  5.1.1). 

5.16.1  Software  quality  assurance  evaluations.  The  developer  shall 
conduct  on-going  evaluations  of  software  development  activities  and  the 
resulting  software  products  to: 

a.  Assure  that  each  activity  required  by  the  contract  or  described  in  the 
software  development  plan  is  being  performed  in  accordance  with  the 
contract  and  with  the  software  development  plan. 

b.  Assure  that  each  software  product  required  by  this  standard  or  by  other 
contract  provisions  exists  and  has  undergone  software  product  evaluations, 
testing,  and  corrective  action  as  required  by  this  standard  and  by  other 
contract  provisions. 

5.16.2  Software  quality  assurance  records.  The  developer  shall  prepare 
and  maintain  records  of  each  software  quality  assurance  activity.  These 
records  shall  be  maintained  for  the  life  of  the  contract.  Problems  in 
software  products  under  project-level  or  higher  configuration  control  and 
problems  in  activities  required  by  the  contract  or  described  in  the  software 
development  plan  shall  be  handled  as  described  in  5.17  (Corrective 
action). 

5.16.3  Independence  in  software  quality  assurance.  The  persons 
responsible  for  conducting  software  quality  assurance  evaluations  shall  not 
be  the  persons  who  developed  the  software  product,  performed  the 
activity,  or  are  responsible  for  the  software  product  or  activity.  This  does 
not  preclude  such  persons  from  taking  part  in  these  evaluations.  The 
persons  responsible  for  assuring  compliance  with  the  contract  shall  have 
the  resources,  responsibility,  authority,  and  organizational  freedom  to 
permit  objective  software  quality  assurance  evaluations  and  to  initiate  and 
verify  corrective  actions. 

C.  EXTRACT  FROM  DATA  ITEM  DESCRIPTION  DI-IPSC-81433 
(SOFTWARE  REQUIREMENTS  SPECIFICATION) 

3.7  Safety  requirements.  This  paragraph  shall  specify  the  CSCI 
requirements,  if  any,  concerned  with  preventing  or  minimizing  unintended 
hazards  to  personnel,  property,  and  the  physical  environment.  Examples 
include  safeguards  the  CSCI  must  provide  to  prevent  inadvertent  actions 
(such  as  accidentally  issuing  an  "auto  pilot  off  command)  and  non¬ 
actions  (such  as  failure  to  issue  an  intended  "auto  pilot  off  command). 

This  paragraph  shall  include  the  CSCI  requirements,  if  any,  regarding 
nuclear  components  of  the  system,  including,  as  applicable,  prevention  of 
inadvertent  detonation  and  compliance  with  nuclear  safety  rules. 
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